#bridgy 2017-11-28

2017-11-28 UTC
snarfed, [chrisaldrich], [eddie], [kevinmarks], [miklb] and [jackysee] joined the channel
#
[jackysee] Hi, found this in the crawl log, is this a problem I should fix?
#
[jackysee] `2017-11-27 22:14:42.674480 I Demoting original post links because user tag:twitter.com,2013:jackysee doesn't match author tag:twitter.com,2013:keyboardwah`
#
[jackysee] The log is at https://brid.gy/log?start_time=1511820907&key=aglzfmJyaWQtZ3lyFQsSB1R3aXR0ZXIiCGphY2t5c2VlDA
[tantek], [cleverdevil] and [snarfed] joined the channel
#
[snarfed] hey Jacky! no, that's just bridgy doing its thing with tweets from other people. nothing to worry about.
[jackysee] joined the channel
#
[jackysee] thx
[jeremycherfas] joined the channel
#
[jackysee] Do I need to have `h-feed` at <html> tag for POSSE post discovery?
tantek, [kevinmarks], [miklb] and [snarfed] joined the channel
#
[snarfed] no, the h-feed class can be anywhere
[manton], [eddie], snarfed and [jackysee] joined the channel
#
[jackysee] Should `u-syndication` also in the `h-feed` page for discovery? I only have `u-url` to permalink now and seems discovery is not success
#
[jackysee] E.g. log https://brid.gy/log?start_time=1511864791&key=aglzfmJyaWQtZ3lyFQsSB1R3aXR0ZXIiCGphY2t5c2VlDA
snarfed and [snarfed] joined the channel
#
[snarfed] bridgy should fetch the u-url and look for u-syndication links there. https://brid.gy/about#link
#
[snarfed] I'll take a look at that log
#
[snarfed] ah, looks like you've had at least some u-syndication links in your feed. bridgy skips u-urls then. try the crawl now button to reset it. https://brid.gy/about#domain
snarfed and [jackysee] joined the channel
#
[jackysee] I think the problem maybe that on my front page (http://jacky.seezone.net), I'll display the full latest article which would have `u-syndication`, the web crawler then use this as my feed page, instead of looking for `<link rel="feed" type="text/html" href="https://jacky.seezone.net/archives_all/">` (this page have no `u-syndication`, just `h-feed > h-entry > u-url`) ?
#
[jackysee] though the log did mention it has fetched the `rel=feed`
#
[jackysee] `2017-11-28 17:14:04.487849 D author's rel-feed fetched successfully https://jacky.seezone.net/archives_all/`
#
[jackysee] last crawl log https://brid.gy/log?start_time=1511889265&key=aglzfmJyaWQtZ3lyFQsSB1R3aXR0ZXIiCGphY2t5c2VlDA
#
snarfed ah, ok. bridgy does fetch rel-feeds even if it sees a u-syndication in a feed...*but* it skips fetching u-urls if it sees a u-syndication in a feed
#
snarfed so yeah if you want your current setup to work, that'd be a new feature request
#
snarfed in the meantime, consider omitting u-syndication from the full article on your front page, then bridgy will fetch the u-urls
#
[jackysee] thanks, I will try that.
tantek joined the channel
#
snarfed thanks for your patience!
#
[jackysee] Just tried, seems not work https://brid.gy/log?start_time=1511890526&key=aglzfmJyaWQtZ3lyFQsSB1R3aXR0ZXIiCGphY2t5c2VlDA
#
[jackysee] do I need to get rid of the `h-feed` at the home page too?
#
snarfed no, the log shows that it fetched your u-urls and found syndication links
#
snarfed it's not great at re-processing existing responses after finding new syndication links. that's probably the problem
#
snarfed looking at that log, it didn't find any *new* twitter syndication urls for your last 10 posts
#
snarfed ah, got it. each time it crawls, it only fetches syndication links for your latest 10 posts. looks like the responses on https://brid.gy/twitter/jackysee are all to posts older than that. so hopefully it should work now going forward.'
[keithjgrant] joined the channel
#
[jackysee] My setting is that the home page only shows articles. So the real feed that contains all the things (notes, links, articles) is in http://jacky.seezone.net/archives_all , which I hope brid.gy would crawl.
#
snarfed right! it does, since you have a rel-feed link to it
#
snarfed it just doesn't go back in history very far. only the last 10 h-entries per crawl
#
snarfed more details: https://github.com/snarfed/bridgy/issues/771
#
[jackysee] but if it's the last 10 post in https://jacky.seezone.net/archives_all , it should not be only those articles. I've got plenty of links/notes these days.
#
[jackysee] I try to remove `h-feed` from my home page, the new log seems to show it try to use top-level h-entries
#
[jackysee] `2017-11-28 17:44:30.161900 I requests.get https://jacky.seezone.net/ {'headers': {'User-Agent': '...'}}
#
[jackysee] 2017-11-28 17:44:32.652150 D No h-feed found, fallback to top-level h-entrys.
#
[jackysee] 2`
#
[jackysee] Is that brid.gy tries to fetch both home page feed **AND** rel-feed ? So it always stops at home page's 10 articles.
#
[jackysee] My home page has about 15 articles link.
#
snarfed ahh then yes it's probably only looking at those
#
snarfed i can bump the limit up to 50 for your account, like i did for aaronpk in that issue link. i'll do that.
#
[jackysee] wow thanks!
#
snarfed do you plan to use bridgy with any other silos? facebook, etc?
#
[jackysee] But I do really want brid.gy to only read the archives_all page. I will try to drop the `h-entry` from home page also.
#
[jackysee] For article I will also publish to facebook
#
snarfed hmm ok, dropping h-entry should help
#
snarfed if that doesn't work, i'll bump you to 50
#
[jackysee] But article is much less frequent.
[kevinmarks] joined the channel
#
[jackysee] dropped also `h-entry` from home page. Now it fetches the archives_all !
#
[jackysee] https://brid.gy/log?start_time=1511893402&key=aglzfmJyaWQtZ3lyFQsSB1R3aXR0ZXIiCGphY2t5c2VlDA
#
[jackysee] Thank you~
snarfed joined the channel
#
snarfed woo, you're welcome!
snarfed, tantek, [keithjgrant], [eddie] and [jackysee] joined the channel
#
[jackysee] I may file a feature request on this?
snarfed and [miklb] joined the channel