Web Crawler Management Platform Crawlab v0.6.2 Official Release

Overview

Crawlab v0.6.2 is the latest iterative version of Crawlab v0.6.x, bringing a series of improvements, including bug fixes, feature enhancements, and enhanced functionality for environment variables.

Changelog

Bug Fixes

• Unexpected database connection
• Task execution command not effective
• Inconsistent task restart execution command
• Inconsistent task queue
• Unable to update Cron expression
• Tasks continue to run after node is disabled
• Task ID not returned
• Unable to download JSON data
• Unable to start SeaweedFS
• Incorrect MaxRunners setting for tasks
• Duplication not effective
• Task stuck due to long logs
• Unable to monitor data source status
• Git pull error
• Application crash when spider does not exist
• Dataset issues

Feature Enhancements

• Default display of latest list data
• Close Runner after task completion
• Batch logging
• Configure log TTL
• More data sources
• Environment variables
• Remove unnecessary buttons

Community

If you find Crawlab helpful for your daily development or your company, please consider starring it on GitHub. If you encounter any issues, feel free to raise them as issues on GitHub. Additionally, you're welcome to contribute to the development of Crawlab. You can also join the Crawlab technical discussion group by adding WeChat account tikazyq1, where you can communicate and discuss with other developers regarding technical development and deployment usage.

References

• Official Website: https://www.crawlab.cn
• Documentation: https://docs.crawlab.cn
• GitHub: https://github.com/crawlab-team/crawlab
• Demo: https://demo.crawlab.cn/