Public | Automated Build

Last pushed: 2 years ago
Short Description
Weibo harvester for Social Feed Manager.
Full Description


A basic harvester for Sina Weibo public post data as part of Social Feed Manager.

Provides harvesters for Sina Weibo API and Html Parser

Harvesting is performed by weibowarc.

The weibowarchtml is just a possible solution for any exist limitation problem. If possible, the weibo harvester will rely mostly on the API calling.


git clone
cd sfm-weibo-harvester
pip install -r requirements.txt


  • Install Docker and Docker-Compose
  • Geting the API_KEY, API_SECRET, REDIRECT_URI, ACCESS_TOKEN, refer to weibowarc
  • Start up the containers
    docker-compose -f docker/dev.docker-compose.yml up -d
  • Running the tests
    docker exec sfmdocker_sfmweiboharvester_1 python -m unittest discover
  • Check the logs
    docker-compose -f docker/dev.docker-compose.yml logs
  • Shutdown all the containers and clear what you have done
    docker-compose -f docker/dev.docker-compose.yml kill
    docker-compose -f docker/dev.docker-compose.yml rm -v --force
    docker rmi $(docker images -q)

Harvest start messages

The necessary information to construct a harvest start message for the weibo harvester.

Search friend timeline harvest type


  • weibo_timeline

API called

  • statuses/friends_timeline

Optional parameters

  • incremental: True (default) or False


  • count for weibos
Docker Pull Command