#dev 2020-05-30

2020-05-30 UTC
flex14, KartikPrabhu, [chrisaldrich], geoffo, gRegorLove, [John_Payne] and test joined the channel
#
vilhalmer
is it possible to specify as part of an h-feed that a reader should fetch the main contents of each post from its canonical url, so the h-feed doesn't have to contain full content?
#
vilhalmer
but the full post will still appear in a reader channel instead of just a stub
#
vilhalmer
context is that my site is currently excessively static, I am hand-writing all the html
#
vilhalmer
so for my feed, I have to duplicate the content of the page which is obviously suboptimal
#
vilhalmer
(eventually I will probably give up on doing this by hand)
[prtksxna] and KartikPrabhu joined the channel
#
aaronpk
Interesting, I hadn't actually considered a feed giving an indication that a consumer should fetch the permalink or not
#
aaronpk
tho that could easily be abused of course
#
jacky
yeah like hot swapping content
#
jacky
I had a ticket for this
#
jacky
irrespective of the inline content, lwa would fetch that content from the u-url
#
jacky
(because I want to show the comment/like/etc info if it's there)
#
jacky
but having some sort of flag for pushing that content would be interesting
[tw2113], nickodd, [jeremycherfas], KartikPrabhu, dckc, deathrow1, moppy and dogfarted joined the channel
#
Loqi
Ok, I'll tell them that when I see them next
#
Zegnat
!tell [LewisCowles] when you are ready for round two of what-the-huh-DOMDocument, my initial testing says almost *any* parsing option will fix the behaviour: https://gist.github.com/Zegnat/a94489e9b7d5501193e724e336bc6052#file-lookingforoptionsresults-txt
[grantcodes] joined the channel
#
Zegnat
cweiske++
#
Zegnat
LewisCowles++
#
Zegnat
gRegorLove++
#
Loqi
LewisCowles has 11 karma in this channel over the last year (23 in all channels)
#
Loqi
gRegorLove has 14 karma in this channel over the last year (82 in all channels)
#
Loqi
cweiske has 5 karma in this channel over the last year (11 in all channels)
[prtksxna] and dogfarted joined the channel
#
@rMdes_
Can't wait for Known to have a "reader" built-in so I can use it with Indigenous micropub/microsub app #indieweb (https://blog.rmendes.net/s/zEdP8)
(twitter.com/_/status/1266708427625172995)
#
vilhalmer
aaronpk / jacky I wonder if it could be indicated by the presence of a p-summary but no e-content within the h-entry?
#
vilhalmer
or neither, for a super-compact feed
#
vilhalmer
would need to go poke around some feeds to see if that would have too many false positives
#
vilhalmer
I've been using kevinmarks' homepage as an example
#
vilhalmer
this could be a toggle within the reader too, I know a lot of rss readers have an option to go fetch the entire page for feeds that only have chunks
flex14 and jeremych_ joined the channel
#
@nickang
↩️ Oh there's an actual web service for this kind of thing → https://webmention.io/
(twitter.com/_/status/1266711386098368513)
[schmarty] joined the channel
#
jacky
vilhalmer: that sounds eerily familiar of a conversation
#
jacky
but that sounds like a reasonable way to do it tbh
#
jacky
what is a h-feed
#
Loqi
h-feed is a microformats2 draft specification with a top level feed object to contain root class(h-*) objects, usually h-entry posts and optionally a common author, name, and representative photo https://indieweb.org/h-feed
#
vilhalmer
emersion of the sway project created a database for drm driver data that people can voluntarily and anonymously upload their information to, it'd be cool to have something similar to quickly explore how common different mf2 objects are in the wild
#
vilhalmer
and easier, since it's all public by design
#
vilhalmer
given a site, pick a feed out and strip all the actual content leaving just the html structure
#
vilhalmer
could use the css selectors syntax to query the database maybe?
#
vilhalmer
weekend project :)
#
jacky
vilhalmer: I'm fori t
flex14 joined the channel
#
jacky
*for it
dogfart joined the channel
#
vilhalmer
I do enjoy writing scrapers…
#
vilhalmer
this weekend is booked but maybe next
[snarfed] joined the channel
#
[snarfed]
vilhalmer i assume you've seen https://indiemap.org/
#
Loqi
Indie Map is a public IndieWeb social graph and dataset. 2300 sites, 5.7M pages, 380GB HTML with microformats2. Social graph API and interac...
#
vilhalmer
no I haven't! this is awesome
#
jacky
thinks we mentioned this before and you mentioned indiemap [snarfed]!
#
vilhalmer
[snarfed]++
#
Loqi
[snarfed] has 51 karma in this channel over the last year (93 in all channels)
#
[snarfed]
hah good, i aim to be predictable
#
jacky
I think a blocker was that indiemap took time to run and wasn't meant to be representative of the 'whole indiemap'
#
vilhalmer
I think if I wrote my proposed thing it would just have a place to paste a url to go scrape rather than crawling the whole map
#
vilhalmer
keep it simple
#
vilhalmer
and it would stay cached pretty much forever, unless I added a new feature that required re-scraping
[KevinMarks] joined the channel
#
[KevinMarks]
You will hit some inconsistencies with "fetch the url if there's no content only summary" from sites like adactio.com or daringfireball.net where they have some posts that are effectively short comments on external links, but I think the ux still works of expanding name -> summary -> content. What you may find is that the linked-to content doesn't have mf2 markup
#
vilhalmer
I wonder if it would be best to limit this behavior to u-uids within the same domain?
#
vilhalmer
I don't think it's generally the intent of linkblogs to have the external content appear directly in their feed
#
vilhalmer
er, u-url not u-uid
#
[KevinMarks]
The difference may be if you iframe the page rather than parsing it?
[LewisCowles] joined the channel
#
[LewisCowles]
[Zegnat] it won't, I tried a few flags before settling on the one I PR'd ;)
#
Zegnat
[LewisCowles]: are you sure? I tested it using the PHP container you linked
#
Zegnat
If you have a moment / have the PHP env still available, I would love to see your output for https://gist.github.com/Zegnat/a94489e9b7d5501193e724e336bc6052#file-lookingforoptions-php
#
Zegnat
(Three example outputs with their PHP versions also in that gist)
#
[LewisCowles]
The no warning flag might work. It's not one of the ones I tested.
#
[LewisCowles]
Scrolling way up there was another libxml flag I tried which did not work
#
[LewisCowles]
I wonder if we can find a branch in the PHP extension that uses your flag or mine
#
Zegnat
Flags are tricky, also because their availability depends on both PHP and libxml version.
#
Zegnat
Which should all be handled by that PHP file
#
Zegnat
I settled on the no warnings flag because a) it worked everywhere I tested, and b) it should have no effects on php-mf2 as we already surpress errors and warnings.
#
[LewisCowles]
It's a fine flag if it passes. I think I may have mis-interpreted what you were saying as "any `DOMDocument->load($string, $flag)` works. You mean most LIB_XML flags apart from `LIBXML_NOBLANKS` right?
[chrisaldrich] joined the channel
#
Zegnat
And LIBXML_NOBLANKS is doing its actual job by removing text nodes that are only whitespace :P
#
Zegnat
LIBXML_PARSEHUGE, the one your PRd, is not available to libxml < 2.7.0. I do not think that matters, but I have no way of testing unless we go hunting for an even more obscure PHP build, hahaha
#
Loqi
hehe
#
[LewisCowles]
Which ones are using < 2.7.0? I only tested it locally in my docker + travis
#
[LewisCowles]
I'm fine with not using that flag
#
Loqi
Zegnat has 22 karma in this channel over the last year (55 in all channels)
#
Zegnat
Yeah, none of our test cases have been using < 2.7.0 so far.
#
Zegnat
But I do not know what some shared-hosting-WordPress-instance is running on, is my thinking.
#
[LewisCowles]
It's solid compassionate thinking
#
Zegnat
We could have also switched container on Travis, probably, and “fixed” it that way. But that would not have helped people like gRegorLove who actually got the problem on their DreamHost account.
#
Zegnat
So I think I picked the right flag for the job. But of course, please scrutinise the PR to the fullest!
#
[LewisCowles]
oh that could be me too. I keep a dreamhost around and it's what I use GWG work on
#
[LewisCowles]
The PR looked fine. I've not contributed so, I don't think I can give any meaningful review. Your work seemed good.
#
[LewisCowles]
Peter / Chun-Sheng has another open PR. I think it should wait until after your one merges, but they are a nice person that has contributed to some other PHP I've done in the past
#
GWG
[LewisCowles]: What is GWG work?
#
[LewisCowles]
their GitHub is Peter279k, their actual name is not Peter. I'll never cease to be amazed at pseudonyms that are actually names
#
[LewisCowles]
I have a test WP which has a few WordPress plugins for IndieWeb. You're my reference point on how some things should work
#
[LewisCowles]
Weather plugin tests were all on a DreamHost
#
[LewisCowles]
I think you use php-mf2 don't you GWG?
#
GWG
I do, for Parse This, which takes the output and processes it for Post Kinds and Yarns
#
GWG
I would like to put it into more
#
GWG
Webmentions is getting it
#
Zegnat
[LewisCowles]: yeah, I think I have seen drive-by PRs from him on other projects. They are good. Just not sure about that PR because it seems itterative without goal. Preparing for future PHPUnit? Better to just get that update in completely I think.
#
Zegnat
But I am focussing on getting the tests merged first, then I’ll leave a proper review on his :)
#
[LewisCowles]
Oh he's always doing those sorts of PR's, but occasionally I've ignored them and then needed to manually do the work later
#
[LewisCowles]
Blindly using latest serves little benefit, but how far behind are we. Some of the smaller changes in visibility and function signatures seemed solid
#
Zegnat
We are on a defunct version of PHPUnit, because there was a need to support old PHP that newer ones do not support running on.
#
Zegnat
Now with the PHP 5.6 bump merged, we can upgrade to a more recent version of PHPUnit.
#
Zegnat
Even better would be to support multiple versions of PHPUnit, so we can run one for PHP 5.6 and one for PHP 7+
#
Zegnat
That is what I have seen multiple other projects do. And would probably be better than just changing the test signatures and not actually changing the PHPUnit version
#
[LewisCowles]
It seems like an unfortunate smell
#
[LewisCowles]
i.e. what you want is a wide range of PHP with test coverage for each
#
[LewisCowles]
but the way of getting it is hacking phpunit version
#
[LewisCowles]
It's documented, so it's not wrong
#
[LewisCowles]
but is it not possible to use composer.json and let travis pick the right version?
#
Zegnat
Hacking version? I think we can just tell composer to get version "^5|^8" and we would have full coverage?
#
[LewisCowles]
that would then defer the lock file creation to the tests
#
[LewisCowles]
I'd never seen that notation
#
Zegnat
You can have ORs in semver numbers for matching. When you run composer it will figure out the dependency tree and grab the right one for your platform / the right one to match your other dependencies.
#
[LewisCowles]
TIL: Zegnat++
#
[LewisCowles]
I'm not sure I'll remember it. Will it cope with phpunit changing signatures?
#
Zegnat
As long as the test cases are supported by both 5 and 8 we would not need to have any duplicate codes or hacks to support it.
#
Zegnat
That is kinda on PHPUnit :(
#
[LewisCowles]
even then the "hacks" can be stripped into checks in a base test class, so you extend that instead of PHPUnit, and handle changes there
#
[LewisCowles]
Nothing is impossible, but the effort...
#
Zegnat
I think the only thing that you need to check for is the class your tests are extending
#
Zegnat
PHPUnit 5 was still using PHPUnit_Framework_TestCase and I think PHPUnit 8 may have dropped that in favour of namespaced classes
#
Zegnat
peter279k has fixed this before though, it is all pretty straight forward, just something you need to take some time out for to test: https://github.com/paragonie/random_compat/pull/163
#
Loqi
[peter279k] #163 Fix Travis CI build
#
[LewisCowles]
See, he's a nice guy
#
Zegnat
Yeah, I knew I had seen him in other projects :D That is not the problem, hehe
#
[snarfed]
vilhalmer: eg re "explore how common different mf2 objects are in the wild" specifically, check out https://indiemap.org/docs.html#mf2_by_sites
leg and [jgmac1106] joined the channel
#
Zegnat
Interesting extension for checking on site performance: https://addyosmani.com/blog/web-vitals-extension/
nickodd, [John_Payne], [prtksxna], [schmarty], flex14, geoffo, [snarfed], KartikPrabhu, [KevinMarks], oenone and gRegorLove joined the channel; nickodd left the channel
[chrisaldrich], [John_Payne] and xsteadfastx joined the channel