#dev 2017-08-21

# KartikPrabhu joined the channel
#
Zegnat
sknebel, you might want to take a look at the PHP safe URL lib I linked there as well. I believe they have a Python implementation of the same thing too
#
Zegnat
It is interesting, and definitely something worth thinking about with the amount of untrusted URLs we access with indieweb tech
# [pfefferle] joined the channel
# jeremycherfas joined the channel
#
jeremycherfas
Wow, sebsel. I am looking forward to trying gimme-a-token later on today. Thanks.
#
Loqi
jeremycherfas: sebsel left you a message 17 hours, 4 minutes ago: I have his helper for obtaining an access token for you now :) https://gimme-a-token.5eb.nl/
#
Loqi
sebsel has 10 karma in this channel (42 overall)
#
Loqi
[IncludeSecurity] safeurl-python: Python implementation of SafeURL
#
cweiske
you could use shpub to get a) curl commands for common micropub tasks and b) a token
#
jeremycherfas
What is shpub
#
Loqi
shpub is a command line micropub client written by Christian Weiske https://indieweb.org/shpub
#
jeremycherfas
Thanks cweiske — I will check that out too.
#
jeremycherfas
I’m beginning to think, with all the cool tools people are sharing, that it might be interesting to attempt to build my own workflow that would look at an RSS feed every, say, six hours and then post any differences to my Known site.
# [colinwalker] joined the channel
#
cweiske
I actually had a script that posted instagram posts via micropub for archiving
#
sknebel
Tools << helper for obtaining an access token https://gimme-a-token.5eb.nl/
#
Loqi
[[tools]] ! https://indieweb.org/wiki/index.php?diff=40151&oldid=38663&rcid=40060 * Loqi.me * (+71) sknebel added " helper for obtaining an access token https://gimme-a-token.5eb.nl/" to "See Also"
#
Loqi
ok, I added " helper for obtaining an access token https://gimme-a-token.5eb.nl/" to the "See Also" section of /tools
# [pfefferle] joined the channel
# [colinwalker] joined the channel
# barpthewire joined the channel
# jeremycherfas joined the channel
# eli_oat joined the channel
# singpolyma joined the channel
# dougbeal|iOS joined the channel
# snarfed joined the channel
# loicm joined the channel
# jeremycherfas_ joined the channel
# [cleverdevil] joined the channel
# snarfed joined the channel
#
jeremycherfas
Looking for guidance on how best to check an RSS feed for updates. I can either store the feed from one fetch in a file, and then compare that with the current fetch to find differences.
#
jeremycherfas
Or I can store the time of the most recent item in the fetch, store that, and then check for items more recent that that.
#
jeremycherfas
The second seems more efficient, but I do not know how to store a variable once the program has finished, except by writing it to a file. Is there a better method?
#
Zegnat
The best method might still be https://en.wikipedia.org/wiki/HTTP_ETag which would allow you to ask a server “is the RSS feed you have any newer than the one I have already seen?”
#
jeremycherfas
Right, that would be handy in deciding whether to fetch the feed in the first place. But if I am going to POST only newer items, I also need to know which items are old, IYSWIM
#
Zegnat
Aah, yeah, I don’t think there is actually a fool-proof way for that unless you can make assumptions about the source RSS.
#
jeremycherfas
So then probably writing the time of the most recent item to a file and then using that to decide whether to post other items seems like a good approach.
#
Zegnat
E.g. items within the RSS do not need to be sorted, and of course old ones can be updated.
#
Zegnat
If you need a catch-all, like most readers, there is no choice but to walk through the entire file every time and compare their UIDs + dates.
# [colinwalker] joined the channel
#
Zegnat
If you are sure the RSS will never get an item inserted later with a slightly earlier date (e.g. because sourced from some silo), yes
#
jeremycherfas
That seems the way I will go then.
# [kevinmarks] joined the channel
#
[kevinmarks]
etag and if-modified since are good
#
[kevinmarks]
what do you code in, jeremycherfas ?
#
snarfed
i'd actually recommend just storing and comparing seen UIDs, nothing else. (if an entry's date, content, etc change but the UID is the same, it's still the same post, just with e.g. a typo fixed, so if you post it again, it'd be a dupe)
#
[kevinmarks]
ah, OK. Python feedparser has etag support built in
#
Loqi
HTTP Conditional Get for RSS Hackers
#
jeremycherfas
Thanks snarfed. I can try that too
#
[kevinmarks]
yes, do what snarfed says first; the etag is a (useful) optimisation to your polling
#
[kevinmarks]
as is websub…
#
Loqi
[[Homebrew Website Club]] ! https://indieweb.org/wiki/index.php?diff=40156&oldid=40118&rcid=40065 * Manton.org * (+36) Updated possible venues. Still looking.
# KartikPrabhu joined the channel
# snarfed joined the channel
# j12t joined the channel
# snarfed joined the channel
# jeremycherfas joined the channel
# snarfed joined the channel
# [cleverdevil] joined the channel
# snarfed joined the channel
#
Loqi
[[Main Page]] ! https://indieweb.org/wiki/index.php?diff=40158&oldid=40039&rcid=40067 * Kartikprabhu.com * (-13) remove #p-personal from display:none to show login stuff on homepage
#
jonnybarnes
how do you leave a message for someone with Loqi
#
jonnybarnes
!tell snarfed deployed p-bridgy-silo-content sections for notes where I mention people and it works as expected in preview, need to write a note that mentions someone and is longer than 140chars to test shortening
#
Loqi
Ok, I'll tell them that when I see them next
#
snarfed
jonnybarnes: yay!
#
Loqi
does a happy dance!
# KartikPrabhu joined the channel
# KartikPrabhu joined the channel