czbin/yidu-spider repository overview

1. ĬΪɼָС˵ ͬ-ca
2. ɼָС˵ʹò -ca
3. ֻɼС˵ ʹ-c 111,222,333 -c 111-222ɼ ǰ˼ΪɼĿվ½ںΪ111,222,333С˵ ˼ΪɼĿվ½ںŴ111222֮С˵
4. ɼгһЩ½ڣ С˵桢顢ȡ ʹ޸ɼ -ra ָ޸
5. ֻ޸С˵ ʹ-r 111,222,333 -c 111-222ɼ -c ֻ-cΪɼָС˵ -rΪ޸ָС˵
6. ʹ-r-ra޸ɼʱͬʱƶҪ޸ѡ -cover,-intro,-top,-sub,-keywords,-degree,-etxt,-txtϲֱӦͼƬ С˵飬 С˵࣬С˵ϸ࣬ ؼʣ дȣ ½(ֻ޸½) txtı(۱ǷڶӦ½ݣ ֻҪָtxt²ɼ)
7. ҪָɼʱʹõĹ ʹ-rule xxxx.xml(ͬʱƶ-ca -c -ra -rĸ֮һ) xxxx.xmlΪrulesļµĹ עIJҪʹļȫ ˺׺ģ ֪ôļ׺ ʹùȸ߶

׶

1. jdbc.properties#for postgresqlµпʼ#ȥͬʱ#for mysqlµ׼#
2. site.iniеlocal_programֵ޸Ϊyidu
3. collect.iniеcreate_htmlΪfalse

ý棺

1. jdbc.properties#for postgresqlµ׼#ͬʱ#for mysqlµ׵#ȥ
2. site.iniеlocal_programֵ޸Ϊjieqi
3. collect.iniеcreate_htmlΪtrue

site.ini: Ҫ local_programtxt_filecover_dir local_programο[׶][ý] txt_fileΪtxtļ···· õʱҪעⲻҪռλɾ cover_dirΪķͼƬĿ¼Ŀ¼Ŀ¼Ŀ¼ õʱעռλ

ɼĬΪִ߳У ж߳ 밴ã

1. ݿΨһ create unique index unique_index_articleno_chaptername ON t_chapter(articleno,chaptername); create unique index unique_index_articlename_author ON t_article(articlename,author);
2. collect.iniеconcurrent_novel_taskΪ1

̲߳ɼƣ ϲɼٶȿ

̲߳ɼգ

1. ĸϵͳԴ
2. ̲߳ɼٶȹ죬 ʱԣ ܻɵܶ½

ͬʱ޸ɼ