aboutsummaryrefslogtreecommitdiffstats
path: root/crawler
Commit message (Collapse)AuthorAgeFilesLines
* Add comprehensive test suite — 81% cross-package coverageAdam Mathes13 days1-0/+233
| | | | | | | | | | | | | | | | | | | | | | Bug fixes: - config: remove unused log import - item: fix Printf format %d->%t for boolean ReadState - util: update stale config.Read -> config.Init, remove config.Config.DBServer Test files added: - config/config_test.go: Init, readConfig, addDefaults (100%) - vlog/vlog_test.go: Printf, Println verbose/silent (100%) - models/db_test.go: InitDB tests - models/feed/feed_test.go: CRUD, filter, Categories, NewFeed, ResolveFeedURL (87%) - models/item/item_test.go: CRUD, Filter with category/search/starred, rewriteImages (71%) - exporter/exporter_test.go: all export formats (91%) - importer/importer_test.go: InsertIItem, ImportJSON (90%) - crawler/crawler_test.go: GetFeedContent, CrawlFeed, CrawlWorker, Crawl (89%) - web/web_test.go: auth, login/logout, stream, item, feed, category, export, crawl, imageProxy handlers (77%) Remaining 0% functions require HTTP/rice.MustFindBox/main entry and can't be unit tested without refactoring (see tickets NK-gqkh96, NK-6q9nyg).
* close results channelAdam Mathes2018-07-071-0/+1
|
* add \n to stored item loggingAdam Mathes2018-07-041-1/+1
|
* spellingAdam Mathes2018-07-041-1/+1
|
* set limit to number of simultaneous crawl jobsAdam Mathes2018-04-301-6/+28
|
* opml exportAdam Mathes2018-04-071-0/+1
|
* remove extra printing in crawlerAdam Mathes2018-03-021-2/+0
|
* custom user-agent for crawlerAdam Mathes2018-02-201-1/+48
|
* turn off full content crawlerAdam Mathes2018-02-191-2/+3
|
* enable full_text and header_imageAdam Mathes2017-11-191-8/+9
|
* silent by default, verbose option added. -feeds cmd line option addedAdam Mathes2017-02-251-7/+8
|
* crawler fixAdam Mathes2017-02-171-2/+1
|
* oops, timestamps againAdam Mathes2017-02-161-2/+4
|
* trying again with date fmtAdam Mathes2017-02-161-3/+5
|
* well that date stuff didnt work right at all whoopsAdam Mathes2017-02-161-3/+3
|
* change to personal namespaceAdam Mathes2017-02-071-2/+2
|
* fix typoAdam Mathes2017-02-021-1/+1
|
* fix crash on import related to unparsed datesAdam Mathes2017-02-021-2/+3
|
* import pathsAdam Mathes2017-02-021-1/+1
|
* fix includesAdam Mathes2017-02-021-1/+1
|
* slightly better handling of datesAdam Mathes2017-02-021-3/+2
|
* switch rss parsers, properly support content:encodedAdam Mathes2017-02-011-5/+24
|
* print item creation errors during crawlAdam Mathes2017-01-241-1/+4
|
* neko v2 initial commitAdam Mathes2017-01-231-0/+63