github binux/pyspider v0.2.0
First Working Release

latest releases: v0.3.10, v0.3.9, v0.3.8...
pre-release9 years ago

Base

  • mysql, mongodb backend support, and you can use a database uri to setup them.
  • rabbitmq as Queue for distributed deployment
  • docker supported
  • support for Windows
  • support for python2.6
  • a resultdb, result_worker and WEBUI is added.

Scheduler

  • cronjob task supported
  • delete project supported

Fetcher

  • a phantomjs fetcher is added. now you can fetch pages with javascript/ajax technology!

Processor

  • send_message api to send message to other projects
  • now you can import other project as module via from projects import xxxx
  • @config helper for setting configs for a callback

WEBUI

  • a css selector helper is added to debugger.
  • a option to switch JS/CSS CDN.
  • a page of task history/config
  • a page of recent active tasks
  • pages of results
  • a demo mode is added for http://demo.pyspider.org/

Others

  • bug fixes
  • more tests, coverage is used.

Don't miss a new pyspider release

NewReleases is sending notifications on new releases.