#dev 2018-03-13

2018-03-13 UTC
snarfed and eli_oat joined the channel
#
aaronparecki.com edited /reader (+149) "+Monocle" (view diff)
#
[cleverdevil] Having a channel automatically load from last unread spot would be awesome.
#
[cleverdevil] For things like my Micro.blog and Twitter timelines.
#
aaronpk oh like not starting at the top?
#
[cleverdevil] Yup.
#
aaronpk hm I think that will require some server work
#
[cleverdevil] Not necessarily.
#
aaronpk there might be several pages of unread items tho, and the client doesn't know where they end
#
[cleverdevil] But it would likely make it easier for clients if the server helped.
#
[cleverdevil] Right. It’d have to keep fetching until it hit an unread item.
#
aaronpk I guess it could do that!
snarfed joined the channel
#
loqi.me edited /Monocle (+114) "aaronpk added "[https://aaronparecki.com/2018/03/12/17/building-an-indieweb-reader Building an IndieWeb Reader] by {{aaronpk}}" to "See Also"" (view diff)
#
loqi.me edited /Aperture (+114) "aaronpk added "[https://aaronparecki.com/2018/03/12/17/building-an-indieweb-reader Building an IndieWeb Reader] by {{aaronpk}}" to "See Also"" (view diff)
snarfed and eli_oat joined the channel
#
grantcodes Ok infinite scroll deployed! Just need to enable it in each channel settings. Settings are a bit broken but works well once that's done
#
grantcodes aaronpk: does your site repeat attempting to send homepage mentions if they don't work? I've not set anything up yet for saving homepage mentions
tantek, [snarfed], [kevinmarks] and leg joined the channel
#
aaronpk grantcodes: no I don't have any retry logic yet. Tho any time I get a like or comment on that post it'll send the webmentions again!
eli_oat and [cleverdevil] joined the channel
#
[cleverdevil] Nice work grantcodes! One thing: we should add a confirmation to channel deletion. It’s a bit too easy to accidentally tap 😀
tantek and tbbrown joined the channel
#
@Inkwater_Masha Too Scared To Write? The One-Two Punch Approach That Finally Got Me Writing http://ow.ly/S4BO30iFGdp #writetip #amwriting #indieauth (twitter.com/_/status/973384411709366272)
renem and [eddie] joined the channel
#
[eddie] Lots-of-notifications.jpg https://files.slack.com/files-pri/T03QR2B2T-F9NPET97W/image_uploaded_from_ios.png?pub_secret=ff1cbdfee5&name=Image uploaded from iOS.png
#
[eddie] I need to find a way to de-dupe my webmentions
#
[eddie] 😆😆
#
[eddie] aaronpk you’ve got a lot of interest in that post 😉
#
[eddie] 😆
#
tantek [eddie] clustering?
#
aaronpk Oh no haha
#
Loqi awesome
#
aaronpk not deduping on source url?
#
aaronpk Are you using the Micropub API for that channel?
#
AngeloGladding hey guys i'm trying to do something that seems like it requires a two-step webmention but i don't think that's right.. before i go any further down this path anyone care to give an opinion?
#
dansup AngeloGladding: explain ...
#
AngeloGladding Alice has a software project and Bob wants to create an issue..
#
AngeloGladding Alice's project is here: https://dev.alice.lahacker.net/code/canopy
#
Loqi [Alice Anderson] canopy
#
AngeloGladding Bob clicked the webaction icon for adding an issue (the exclamation) and was taken to his site where he created an issue: https://dev.bob.lahacker.net/issues/2018/03/13/4afi/remove_static_assets
#
Loqi [Bob Brown] Remove static assets from repository
#
AngeloGladding the issue is in-reply-to the project
#
dansup how so?
#
AngeloGladding if you follow the second link you'll notice a "reply" icon and hyperlink back to Alice's project
#
AngeloGladding it's denoted as "in-reply-to" which I believe is similar to brid.gy's support of GitHub POSSE
#
AngeloGladding Bob sends a webmention to Alice and the issue is presented in the column of the project's page
#
AngeloGladding what i'm wanting to do..
#
AngeloGladding oh and upon receipt of the webmention Alice also created: https://dev.alice.lahacker.net/code/canopy/issues
#
Loqi Alice Anderson's tree
#
AngeloGladding https://dev.alice.lahacker.net/code/canopy/issues/1
#
Loqi Alice Anderson's tree
#
AngeloGladding what i'm wanting to do is get that last link presented on Bob's issue page
#
AngeloGladding with a "*syndication*" property set
#
AngeloGladding so what i'm currently thinking is to have Alice send an additional webmention back to Bob's issue referencing the "canonical" issue page on Alice's site
eli_oat joined the channel
#
@codebear @lmorchard there are some thoughts on using webmentions and h-feed for that over at Indieweb (twitter.com/_/status/973423650723463169)
snarfed, [asuh], AngeloGladding and cweiske joined the channel
#
cweiske AngeloGladding, using webmentions for issue tracking is a great idea
tantek and [kevinmarks] joined the channel
#
[kevinmarks] That sounds like a salmention approach - the kind of comment threading that some people have already built
iasai and [kevinmarks] joined the channel
#
[kevinmarks] https://developer.mozilla.org/en-US/docs/Web/CSS/grid-auto-flow grid-auto-flow: dense looks like it might be an answer to the image packing stuff Kartik and Aaron were iterating on
#
[kevinmarks] nice indieauth explanation here: https://medium.com/samsung-internet-dev/raindrops-on-roses-whiskers-on-kittens-declarative-web-apis-and-truly-serverless-web-endpoints-50d733e6d867
[tantek] joined the channel
#
[tantek] The always insightful cwebber https://twitter.com/dustyweb/status/973239603942608896
#
@dustyweb If you have a text based format you want humans to look at and occasionally write by hand and you don't permit comments, you're making a terrible mistake. JSON, I'm looking at you. (Markdown, I'm also looking at you for having ugly comments as an extension.) (twitter.com/_/status/973239603942608896)
#
[tantek] Beautiful 1.5d / almost fit-on-a-line transit visualizations, perhaps inspiration for mass transit /travel post designs, especially denser in-stream displays: https://twitter.com/sftru/status/972195929661952001 (also thread)
#
@SFTRU Without NextBus to hide our poor frequency, this feels like a good time to remember what @sfmta_muni frequency looked like in the 1920s. Only the D & F lines had frequencies worse than every 4 minutes (every 4.5 min). The 7-Haight ran every 2 minutes. https://pbs.twimg.com/media/DX3vB7NVwAAyo4R.jpg (twitter.com/_/status/972195929661952001)
#
[tantek] I feel like Tufte would have a name for these
jeremycherfas, barpthewire, leg, eli_oat, globbot and snarfed joined the channel
#
dgold chatting with cweiske elsewhere, we think that the mf2-parsers aren't handling indentation whitespace correctly in h-feed
#
dgold e.g. my site entries are shown with trailing \n and \t
#
dgold but all those are are either emty lines or indentation to the closing </div>
#
aaronpk parsers definitely do not agree on whitespace handling right now https://pin13.net/mf2/whitespace.html
#
aaronpk the "expected" column is my own expected result, not necessarily what the spec actually says right now, since i'm also not convinced the spec says the right thing yet
#
dgold i did not know that was there, thanks aaronpk
#
aaronpk I made that last week when we were having a similar conversation
#
dgold :) monocle throwing these things up?
#
aaronpk indeed
#
aaronpk it's nice having a sudden increase in mf2 consumers compared to just the webmention receiving that we've had so far
#
dgold i'm seeing worse than that on my own site, due to higher indentation levels
#
dgold give me a mo and I'll gist it
#
piperhaywood.com created /User:Piperhaywood.com (+112) "Created page with "Piper Haywood is a developer with a background in fine art. [https://piperhaywood.com https://piperhaywood.com]"" (view diff)
[kevinmarks] joined the channel
#
[kevinmarks] Is that generated dynamically, aaronpk?
#
[kevinmarks] I'm wondering if we can revive the comparison engine in the tests repo
#
[kevinmarks] Or replace it with a CI version
#
aaronpk [kevinmarks]: I have a script that hits all the parsers with the test data, but it takes a while to run
[Mike], snarfed and eli_oat joined the channel
#
piperhaywood.com edited /User:Piperhaywood.com (+3) (view diff)
snarfed and [miklb] joined the channel
#
[miklb] someone please talk me out setting up aperture on heroku today 🙂
#
aaronpk oh gosh
#
[miklb] FOMO is getting the best of me
#
snarfed in related news, cloudflare workers launched (ie graduated to GA) today! https://blog.cloudflare.com/cloudflare-workers-unleashed/
#
snarfed is currently deciding between that and Apigee for trying the microsub bridge with minimal code
#
aaronpk damn, I thought I had solved my paging issues but just encountered a sneaky edge case and now I am reconsidering this entire approach
#
snarfed heh
#
aaronpk I may need some help sorting through this logic. I don't understand why it's so confusing.
#
snarfed ah, damn, cloudflare workers have short deadlines (5-50ms). suitable for redirecting to another API, but not wrapping it. ah well.
#
snarfed aaronpk: happy to help if i can
#
aaronpk I think this is only a problem because I am limited to 1-second resolution for these timestamps
#
aaronpk so entries have their own published timestamp, and then there is a separate timestamp of when they were added to a channel
#
aaronpk when retrieving a page of a channel's timeline, I sort by the added timestamp rather than the entry's published date
#
aaronpk the idea is to treat it like a chat log, new stuff is always added to the end
#
snarfed (ooh actually the 50ms cloudflare worker limit is only CPU, not wall clock time, so may work after all)
#
aaronpk the problem is when a bunch of entries get added to the channel all within the same 1-second interval, they all share the same timestamp, so then a <= comparison no longer works right
eli_oat joined the channel
#
snarfed agreed, second resolution seems insufficient
#
aaronpk I tried disambiguating those by using the published date of the entry
#
snarfed you might also reconsider the "always append" semantics though
#
snarfed eg my reader (newsblur) allows reversing order, which i care about, but i don't really care if new entries get interpolated or are always at the end
#
aaronpk but it turns out rss feeds are particularly low-res enough that the cases where a bunch of entries are discovered at the same time they often also share the exact published date
#
snarfed maybe personal though
#
aaronpk (I do have a special case when you add a new feed, those entries get interpolated based on their published date so that it doesn't flood the channel with new posts)
#
snarfed apart from subsecond resolution, could you preserve order for items that come from the same feed fetch?
#
snarfed ie their order within that feed file
#
aaronpk like storing that as a separate column?
#
snarfed got me, sure
#
snarfed implement however you want
#
aaronpk hm that might work
#
aaronpk that would solve the problem of trying to use the published date to disambiguate because the published date is definitely less distinct than the other timestamp
#
aaronpk What I'm still conceptually missing is how to write the SQL where clause to limit the results properly
#
snarfed heh
#
snarfed try first defining your ordering tuple - ie something like (channel, timestamp, order within source feed) - and then bound that?
#
snarfed i may not fully understand the result limiting need though
[kevinmarks] joined the channel
#
[kevinmarks] that is always hard. You need to make the next/prev links pass in an absolute offset of some kind, so that adding new entries down't throw it off
#
aaronpk right this is all based on absolute references rather than sql's limit/offset where the pages can change
#
aaronpk the absolute reference to an entry is the two timestamps, but i'm changing that to (timestamp,batch_order)
#
aaronpk the paging reference string used in the API is an encoded version of that
#
aaronpk so a client will ask for entries after (timestamp,batch_order)
#
aaronpk one sec, I can post a screenshot to make this easier to see
#
aaronpk https://media.aaronpk.com/Screen-Shot-2018-03-13-07-54-47-GMKDBlYCZ5.jpg
#
aaronpk so when a bunch were added to the channel at the same timestamp, they'll have a distinct `batch_order` value now
#
[kevinmarks] wow atom standardised this wrong https://tools.ietf.org/html/rfc5005
#
aaronpk so let's say for example the first page ends at entry_id=5 in that example, the paging ID returned to the client will be encoded(2018-03-13 07:53:26, 1)
#
aaronpk now I need to select from the database the next page that starts at (2018-03-13 07:53:26, 1)
#
snarfed got it
#
[kevinmarks] and you need an index on that so it doesn't tablescan
#
aaronpk index later. gotta get teh query right to start
#
aaronpk https://media.aaronpk.com/Screen-Shot-2018-03-13-07-57-38-vJaqsI8X5v.jpg
#
aaronpk I can't use "where batch_order < 1" because that would skip the entries lower down where there might be a batch_order=3 from a different batch
#
snarfed is it WHERE added_to_channel > 2018-03-13 07:53:26 OR (added_to_channel = 2018-03-13 07:53:26 AND batch_order > 1)
#
aaronpk aha that might be it
#
snarfed wms aaronpk from a new h-invoice post type
#
aaronpk ha
#
snarfed gotta run, gl!
#
[kevinmarks] or just have a sequence number for channel entries?
#
aaronpk ah crap that's not quite right
#
aaronpk [kevinmarks]: how do you insert entries retroactively with a sequence number?
#
aaronpk arbitrary precision floating point?
#
[kevinmarks] BASIC line numbers / Cupertino street addressses
#
[kevinmarks] you want to fake out the 'added to channel' date?
#
aaronpk yeah for the case when a new feed is added in order to insert those back in time
#
aaronpk heh I remember re-ordering BASIC line numebrs
#
aaronpk that sounds like nightmare logic to write tho
#
[kevinmarks] that is a bit odd though - you don't want to lie about crawl time, do you?
#
kaja.sknebel.net edited /User:Kaja.sknebel.net/upcoming-hwcs (+308) "update from /Events" (view diff)
#
[kevinmarks] I'd have thought you'd want published time to determine that
#
aaronpk published time kinda sucks it turns out
#
[kevinmarks] which I know is a bit of a slippery concept
#
[kevinmarks] yeah
#
aaronpk this endgadget feed updated with 4 entries with the same exact published time of exactly 9:00am
#
aaronpk not helpful
#
aaronpk but yes for newly added feeds, I use the published time to determine where in the channel history to insert the posts
#
aaronpk it was either that or not add existing entries from the feed and only start adding new ones
#
aaronparecki.com edited /Special:Log/rights () "changed group membership for User:Martymcgui.re from (none) to sysop" (view diff)
#
[kevinmarks] the exported twitter archives give all tweets older then somewhere in 2010 a timestamp of midnight
#
[kevinmarks] which is annoying
#
[kevinmarks] enough places get their TZ wrong that published time can be very skewed
eli_oat joined the channel
#
[kevinmarks] this is kind of why ryan ended up building snowflake
#
aaronpk this would be simpler if I only did append-only
#
aaronpk snowflake doesn't let you backdate things either
eli_oat joined the channel
#
[kevinmarks] true, it was for a central notion of truth
#
[kevinmarks] the other approach may be to have epochs based on published time, so you append old entries to those epochs (like tantek's BIMs), but then you need to cross epochal boundaries
#
martymcgui.re edited /MediaWiki:Sidebar (+5) "update text for next-hwc" (view diff)
#
aaronpk I think I have something working based on snarfed's suggestion
#
martymcgui.re edited /next-hwc (+0) "Redirected page to [[events/2018-03-21-homebrew-website-club]]" (view diff)
#
[kevinmarks] with technorati we had recency based shards, so when you searched for keywords it could work back through the recent ones first before going deep if it was a rarer word
eli_oat and snarfed joined the channel
#
aaronpk soooo close snarfed
#
snarfed the sql?
#
aaronpk thanks for the nudge, I think that direction is right
#
snarfed great!
#
aaronpk it's working in both directions, but not with "before" *and* "after"
#
snarfed ah heh
#
aaronpk oh it's just the sql builder that's getting the clause grouping wrong
#
aaronpk sometimes these ORMs just get in the way
#
dgold aaronpk: ah - that explains a query I had as to why stuff was appearing in odd order in readers from aperture
#
aaronpk possibly
#
aaronpk this bug only happens in very specific cases
AngeloGladding and eli_oat joined the channel
#
MylesBraithwaite 👋, I'm currently in the process of developing my own IndieWeb application. Would it be okay if I create a Wiki page in the IndieWeb site for my notes? Or is that only for completed projects?
#
aaronpk absolutely!
#
aaronpk lots of us like to take notes on in-progress stuff on our user pages
#
KartikPrabhu MylesBraithwaite: also add yourself to /irc-people
snarfed joined the channel
#
aaronpk but if your project has a name then you can of course use the project's page as well
#
KartikPrabhu and welcome
#
MylesBraithwaite @KartikPrabhu thanks. I already added my self to the /irc-people page.
#
KartikPrabhu nice
#
KartikPrabhu MylesBraithwaite: you can create a project page to take notes. I have mine https://indieweb.org/Bundle
#
aaronpk dgold: just pushed the paging fix
#
dgold danke.
#
aaronpk requires a migration
#
aaronpk lemme know if you see any more weirdness. this was a pretty significant change
#
dgold there's someone on micro.blog has issues with watchtower's cron job
#
dgold will do aaronpk
#
aaronpk just saw that! hmm
#
dgold aaronpk++ for implementing fixes on an experimental thing
#
Loqi aaronpk has 124 karma in this channel (1587 overall)
#
aaronpk your suggestion of running the cron by hand is good, i'll wait to see what he says to that
#
mylesb.ca created /User:Mylesb.ca/Amalfi (+1302) "Created page with "'''Amalfi''' is an IndieWeb application built using [[Python]] and [[Flask]] that is currently in development by {{Myles}}. People using it on their own site: * {{Myles}} |..."" (view diff)
#
mylesb.ca edited /User:Mylesb.ca (+52) (view diff)
tantek and eli_oat joined the channel
#
KartikPrabhu ancarda: maybe file an issue with Linode?
#
ancarda Hmm, I can try
#
tantek what is a DNS TXT record
#
Loqi It looks like we don't have a page for "DNS TXT record" yet. Would you like to create it? (Or just say "DNS TXT record is ____", a sentence describing the term)
#
ancarda I have no idea why it's 15 minutes
#
ancarda what is DNS records
#
Loqi It looks like we don't have a page for "DNS records" yet. Would you like to create it? (Or just say "DNS records is ____", a sentence describing the term)
#
ancarda (may be listed)
#
ancarda Oh dear
#
ancarda what is DNS
#
Loqi domain name server is an abbreviation for Domain Name Server and often used to refer to the configuring thereof https://indieweb.org/DNS
#
KartikPrabhu DNS records is /DNS
#
Loqi ok
#
loqi.me created /DNS_records (+16) "prompted by ancarda and redirect added by KartikPrabhu" (view diff)
#
ancarda https://indieweb.org/DNS#DNS_Record_Types -- it's listed here
#
sknebel likely because they have a master server somewhere and sync between the others
#
sknebel worth a try how bad it really is
#
ancarda ^^ yes, IIRC the message is "Records update with our DNS server every quarter hour" or something to that effect
#
ancarda Or "our Master DNS server", I can't remember
#
sknebel and I wouldn't be surprised if they have a reasonable tolerance for delay in there
#
dgold how would wildcard certs work for sites which aren't on the same server?
#
dgold e.g. ascraeus.org is on one server, but my plethora of subs are on another
#
sknebel you copy it between servers, or get multiple ones if the number is low enough, or have one server use fixed ones and one a wildcard
#
www.svenknebel.de edited /DNS (+59) "/* DNS Record Types */ AAAA" (view diff)
#
dgold aaronpk: hitting some composer-lock issues with the latest build of aperture
[eddie] joined the channel
#
[eddie] !tell swentel regarding tokens, etc. not optimal but how Indigenous handles it currently is when you log out, it will send a token revocation request. Besides that, it assumes tokens are valid, but will provide an error if your Micropub request has an authentication error. It is assumed for now that if you have an authentication error, you’ll log out and back in of Indigenous manually
#
Loqi Ok, I'll tell them that when I see them next
KartikPrabhu joined the channel
#
aaronpk dgold: what php version? I run it in 7.2
#
dgold "Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. Run update to update them."
#
aaronpk Oh weird
#
dgold that's with a fresh git pull
#
aaronpk Oh I bet I need to commit my composer.lock even though only the hash changed
#
aaronpk Try now
#
dgold saul goodman
#
dgold its all good
#
dgold (its still downloading them, but it is now working happily without remonstrating with me)
#
dgold did you change the scrolling in monocle? it seems smoother...
KartikPrabhu joined the channel
#
aaronpk No, it’s just browser scroll
#
dgold soooo many missed posts :)
#
aaronpk Whoa really? What sort of sources? That should only have happened in very specific cases like when a large batch of entries was suddenly found and they had drastically different published dates
#
aaronpk I’ve only had one instance of that bug in all my feeds
#
dgold it was when I started using aperture - don't worry, I saw them elsewhere
#
dgold but, for the record, 100-odd posts in one channel, 30 in the other
#
dgold monocle is seriously nice
eli_oat joined the channel
#
aaronpk Weird!
renem joined the channel
#
ravisagar.in edited /POSSE (+428) "/* IndieWeb Examples */" (view diff)
#
ravisagar.in edited /POSSE (-4) "/* Ravi Sagar */" (view diff)
snarfed, leg, [kevinmarks], tantek and [miklb] joined the channel
#
aaronpk !!! https://publog.stuifzandapp.com/posts/304
#
Loqi [Peter Stuifzand] I have been buildling a microsub server. It's not perfect, but it works. It works with Monocle. The code is open source and can be found on Github here: https://github.com/pstuifzand/microsub-server/ But sadly it seems I can't use my own authorizatio...
snarfed and snarfed1 joined the channel
#
pstuifzand hi
#
aaronpk waves
#
pstuifzand aaronpk: do you know why the authorization doesn't work?
#
aaronpk I can definitely take a look
#
aaronpk you wrote an authorization endpoint?
#
pstuifzand yes
#
pstuifzand it seemed to be something that could be done
#
aaronpk yes should be doable. have you been able to use it to log in to any client?
#
pstuifzand yes it works with quill
#
aaronpk like https://micropub.rocks/ or https://quill.p3k.io
#
aaronpk ah okay. what error did you get with Monocle then?
#
pstuifzand missing_authorization_endpoint
#
aaronpk well that's odd
#
aaronpk using https://publog.stuifzandapp.com/ as your URL?
#
pstuifzand yes
#
aaronpk k let me check this out
#
aaronpk aha I see what's going on
#
aaronpk it's failing because your site returns a 500 server error for a HEAD request
#
pstuifzand alright
#
pstuifzand now i have to look
#
aaronpk i'll make that error message better
#
aaronpk alright, at least now it doesn't say it was a problem with the authorization endpoint
[cleverdevil] joined the channel
#
[cleverdevil] Good news, I think I've got two little projects I will be pushing up to GitHub late tonight 🙂
#
[cleverdevil] A Python IndieAuth implementation with a Token and Auth endpoint that uses self-encoded tokens.
#
aaronpk ooh
#
[cleverdevil] And a Python Microsub server that stores things in a tiny embedded JSON database, and uses JSON Schema for validation.
#
sknebel sub or pub?
#
bear cool - that's a higher level of functionality than the python code Ihave for micropub
#
[cleverdevil] I probably have done a massive number of things wrong, but I look forward to sharing them and getting feedback 🙂
#
[cleverdevil] Micro*pub*
#
[cleverdevil] Sorry 😄
#
sknebel thought so, just wanted to make sure
#
[cleverdevil] I'd eventually like to do a Microsub server too. But, the bar for that is much higher, given it involves feeds.
#
Zegnat What do you validate with JSON Schema, [cleverdevil]?
#
KartikPrabhu yeah why the validation thing?
#
Zegnat I recently found myself writing a schema for microformats validation for my Micropub endpoint
#
[cleverdevil] Zegnat: the MF2 JSON data that is posted in.
#
Zegnat We probably did double work then, haha
#
Loqi rofl
#
[cleverdevil] https://github.com/cleverdevil/microformats2/
#
KartikPrabhu so if I store random custon properties it won't validate?
#
[cleverdevil] I'm sure it needs tweaking as well, but it covers all of the bits I found documented on the microformats2 wiki.
#
[cleverdevil] By default, it allows "additional properties".
#
Zegnat https://gist.github.com/Zegnat/65ed9a9fb0546fb8c4aa0c0b790b8a40
#
KartikPrabhu then what is being validated?
#
[cleverdevil] But, it obviously won't check them to ensure that they're correct.
#
[cleverdevil] The documented/standard bits.
#
KartikPrabhu is confused
#
Zegnat Mine has no hardcoded properties, but only allows legal property names, [cleverdevil]
#
[cleverdevil] Cool!
#
[cleverdevil] Maybe we can collaborate?
#
KartikPrabhu is more confused
#
[cleverdevil] Take a look at my project and test suite, @Zegnat.
#
KartikPrabhu goes back to science which is less confusing
#
[cleverdevil] I'm not a JSON Schema expert.
#
Zegnat KartikPrabhu, JSON Schema (like an XML schema) can be used to check the validity of a document. In this case the JSON that you get from parsing microformats from HTML.
#
KartikPrabhu yeah I don't know what "valid" mf2 is
#
KartikPrabhu all properties are optional and random custom properties are allowed
snarfed joined the channel
#
KartikPrabhu so what is an invalid mf2?
#
aaronpk {"content":{"foo":"bar"}} for example
#
[cleverdevil] The properties are *optional*, but if you *do* provide them, they have specific formats.
#
Zegnat you can’t have a property named 💩 just to give an example
#
aaronpk every value has to be an array
#
KartikPrabhu aaronpk: that is never a problem is practice though
#
aaronpk KartikPrabhu: if you're storing JSON it can be a problem for sure
#
[cleverdevil] @Zegnat here is the current output from my project: https://gist.github.com/cleverdevil/c770a4fe2b88252da97e11ec961cf778
#
aaronpk it's not a problem most of the time because if you are using a parser the parser will always return valid mf2 json
#
KartikPrabhu I am storing JSON and html+mf2 for my notes and this "validation" thing has never come up
#
aaronpk are you always storing the json from a parser?
#
KartikPrabhu yes! which is how I would do it anyway
#
Zegnat KartikPrabhu when you are accepting random JSON from a possible untrusted source (e.g. any Micropub client) it is good practice to filter such input and make sure what you are getting is what you expect. Like any other outsider input.
#
[cleverdevil] Yup, what @Zegnat said 🙂
#
KartikPrabhu oh well! I'll cross that bridge when it collapses!
#
aaronpk right, so if you're only storing it from a parser (a trusted source) then you don't need to worry about validating it
#
KartikPrabhu totally made up a nonsensical saying
snarfed joined the channel
#
[cleverdevil] Heh
#
Zegnat Oof, that’s a lot of reading [cleverdevil]. I might take it to bed with me.
#
[cleverdevil] Look at the Python side.
#
[cleverdevil] Its easier to consume 🙂
#
[cleverdevil] And it generates the longer JSON schema output.
#
[cleverdevil] The best thing for it would be to have a massive test suite of valid JSON data to validate.
#
Zegnat Not sure if you win anything by specifying every mentioned property though. I guess it is nice because you get to force the uri format on some of them...
#
[cleverdevil] I guess I'm just a completionist 🙂
#
bear I have tons of python mf2 parser json stored back when I was scanning indieweb sites
#
[cleverdevil] \o/
#
Zegnat I think I made a remote rsvp once, that’s not going to get passed your schema for one, [cleverdevil]. As you only accept 4 fixed strings for rsvp.
#
KartikPrabhu see ^
#
[cleverdevil] nods.
#
[cleverdevil] Is that invalid or not?
#
Zegnat Depends on your server implementation? It isn’t invalid mf2, since there are no real fixed vocabs in mf2
#
[cleverdevil] Got it, then I should likely take that out.
#
Zegnat Your schema seems great for validating some form of strict objects. E.g. you want photo to always be a URL, so your schema tests for that.
#
KartikPrabhu why can't I write a p-photo?
#
Zegnat What generic mf2 is concerned, photo can be anything. Maybe it’ll contain a submicroformat h-photo one day.
#
KartikPrabhu or e-photo
#
Zegnat You can, KartikPrabhu. But if the parsed value in the JSON object isn’t an uri, you will not pass [cleverdevil]’s validation
#
Zegnat Which may be correct, if his server expects URLs for the photo property.
#
[cleverdevil] Again, this was just my first pass based upon my *interpretation* of the documentation.
#
aaronpk I think it's important to keep in mind the context in which the validation is being used
#
Zegnat There is a bit of a conflict between validating generic mf2 objects (which is what my schema tries to do) and validating for an actual usecase (e.g. accepting blobs for Micropub).
#
aaronpk right, the difference between mf2 json syntax validation vs vocabulary validation
snarfed joined the channel
#
aaronpk both of which are important but for different use cases
#
Zegnat I can defintely see why you would want a collection of URLs for the photo property on a micropub server. And from all of the h-entry documentation we have, that’s what [cleverdevil] is specifically validating: URLs.
#
bear I have 19G of website html and the parsed mf2 output if you want :)
#
[cleverdevil] Haha
#
[cleverdevil] Look, I am also willing to admit that this project may be a fools errand 🙂
#
Zegnat It is a great project, [cleverdevil]! I think it just needs a clearly defined scope :)
#
[cleverdevil] But, IMO, if I am eventually going to create my own CMS, I want to know that the MF2 JSON that I store isn't a rats nest.
#
bear some sites I have every day from 2015 to today
#
[cleverdevil] Its normalized.
#
aaronpk any micropub endpoint that accepts mf2 json should validate the mf2 json syntax at the very least
#
[cleverdevil] Sure, the vocabularies? Maybe not.
#
aaronpk otherwise you could get into some trouble if you store JSON that is not mf2 JSON later when you go try to access the data
#
aaronpk i'm a fan of treating the vocabularies in a much looser fashion, accepting anything and rendering what you can
#
Zegnat Actually, I think I found an mf2 example in the micropub spec that didn’t validate against my generic schema... Oh well. Another issue for tomorrow! Nighty-night.
#
[cleverdevil] Fair enough. But, that makes the CMS side *much* more complex.
#
aaronpk but certainly there could be use cases where you'd want to strictly validate the vocabularies too. I just think they are totally different concerns
sebsel joined the channel
#
aaronpk also I wouldn't say "much" more complex... I basically did all that logic in the Monocle templates and they aren't huge
#
[cleverdevil] If anything, I'd prefer to validate in a rigorous way, and then normalize/fix before storing, if I want to accept it.
#
[cleverdevil] Sure, but you're using a bunch of tools that you created 🙂
#
[cleverdevil] Well-behaving tools!
#
[cleverdevil] The very loose definition of mf2 means that as adoption accelerates, its very likely that things will get... messy 🙂
eli_oat joined the channel
#
aaronparecki.com edited /next-hwc (-3) "pdx next" (view diff)
dougbeal|mb1 joined the channel
#
GWG Evening
[kevinmarks] joined the channel
#
[kevinmarks] Can you use the tests repository as something to validate?
#
[cleverdevil] That'd be great.
#
[kevinmarks] The microformats tests repository is a bit neglected, but it should help us converge the parsers
#
[kevinmarks] As it is HTML to JSON mappings
chrisaldrich joined the channel
#
@kevinmarks @kellan The #Indieweb is here for you. Post on your site, accept answers to the questions as webmentions. Or go back to nntp (twitter.com/_/status/973707055386198016)
[asuh] joined the channel
#
@kevinmarks @kellan You can POSSE to twitter, that will get you so much "engagement" . Then the #Indieweb will send you webmentions via http://brid.gy and you get to decide which ones are visible. (twitter.com/_/status/973708117937672194)
[chrisaldrich] joined the channel
#
aaronpk just deployed new php parser to xray
#
aaronpk hope I don't break everything
#
www.svenknebel.de created /via (+1288) "create with dfn, examples, why" (view diff)
#
www.svenknebel.de edited /via (+243) "brainstorming" (view diff)