Skip to content

add Chromium support for dynamic page rendering. #17

@Cash111

Description

@Cash111

Description:

  • Add NewHybridRequest for encapsulating a request that uses Chromium.

Example:

func (e *ExampleSpider) StartRequest(req chan<- *tegenaria.Context) {
	for i := 0; i < 20; i++ {
		for _, url := range e.GetFeedUrls() {
			// 生成新的request 对象
			exampleLog.Infof("request %s", url)
			request := tegenaria.NewHybridRequest(url, tegenaria.GET, e.Parser)
			// 生成新的Context
			ctx := tegenaria.NewContext(request, e)
			// 将context发送到req channel
			time.Sleep(time.Second)
			req <- ctx
		}
	}
}

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions