#dev 2020-11-14
2020-11-14 UTC
[CrowderSoup], expl0iter, [hibs], [tantek] and [KevinMarks] joined the channel
# 
@aaronpk ↩️ Meanwhile, Pingback has nothing to do with Atom/RSS and has always worked with arbitrary web pages. That too was XML for literally no good reason, so that’s been simplified into the Webmention protocol (twitter.com/_/status/1327432691453620224)
# 
@coreload ↩️ XML was just due to the era. It works, especially if the tools do it on my behalf. I deal with it all the time in the world of insurance. "It is what it is."
But you've given me some things to think about re: WebMention, etc. and I appreciate it. Good feedback. (twitter.com/_/status/1327434272551014400)
ShinyCyril joined the channel
# 
KartikPrabhu aaronpk++ very nice discussion on that twitter
# 
[KevinMarks] thread re retro web tech https://twitter.com/LeaVerou/status/1327403615162691585?s=20
# 
@LeaVerou @hdv I wish there was a successor to XSLT. Lots of ugliness, but lots of good ideas there too. (twitter.com/_/status/1327403615162691585)
# KartikPrabhu what is XSLT
# 
Loqi It looks like we don't have a page for "XSLT" yet. Would you like to create it? (Or just say "XSLT is ____", a sentence describing the term)
# [KevinMarks] I'm resisting defining XSLT as "now you have three problems"
# KartikPrabhu lol
j12t joined the channel
# 
jacky I'm looking at https://www.iana.org/assignments/link-relations/link-relations.xhtml (since sl007 linked it)
# [KevinMarks] this is great https://wizardzines.com/zines/css/
# [KevinMarks] is that the obsolete version of http://microformats.org/wiki/existing-rel-values
nickodd joined the channel
reed, geoffo, [Raphael_Luckom] and gxt joined the channel
# 
miklb I really love what Simon is doing with Datasette for his personal data https://simonwillison.net/2020/Nov/14/personal-data-warehouses/
dhanesh and [KevinMarks] joined the channel
# [KevinMarks] Datasette is a lovely piece of work
[jgmac1106] and dhanesh joined the channel
# 
[jgmac1106] I can not find a cd ripper that works anymore on MacOS anyone have recommendations....all my old one's either don't work or have unsigned developer certifications....this will be last Mac I buy I think...don't tell me what I can and can't install...my computer not yours
# 
sebbu https://www.freac.org/ should work
# [jgmac1106] sebbu++ I could not find the setting anywhere, many just not updated for the chipset and the others were throwing the warning
# [jgmac1106] "*“XLD” cannot be opened because the developer cannot be verified."*
# [jgmac1106] I have third party allowed. I will keep searching...all my old rippers no longer work
# [jgmac1106] "Identified developers are registered with Apple and can optionally upload their apps to Apple for a security check" ..getting it on all my old favs
# 
petermolnar CDDB was taken down?!
# [jgmac1106] I feel like if I breakdown and use iTunes Apple wins....plus I would have to open ITunes (or whatever they call it now...I hate that app soooo much)
# [jgmac1106] feedb shuts down every now and again hopefully this time not for good
# [jgmac1106] maybe more permanent: https://en.wikipedia.org/wiki/Freedb
# [jgmac1106] but you can http://gnudb.org/howto.php
# [KevinMarks] iTunes is a decent ripper (or at least used to be, I haven't used it in a while) - fast and good quality.
carolline joined the channel
# [jgmac1106] Yeah Kevin gonna break down and launch it...just think every UX 101 book should be called "How not to Build iTunes"
aaronpk, nolith and MrHyde joined the channel
smacko[m], jeremycherfas, kitt, nolith[m] and dietricha joined the channel
# Loqi RelMeAuth is an authentication method that uses personal URL for identity that rel-me link to established OAuth provider(s) to perform the actual authentication https://indieweb.org/RelMeAuth
[Raphael_Luckom] joined the channel
# [Raphael_Luckom] Feel free to ignore if this is a rtfm question, but what are the use cases for indieauth? Specifically, in a conversation yesterday, [aaronpk] made a pretty good case that you might not need to require AAA on a webmention endpoint specifically, which might mean that you could do a lot of interactions without any auth at all. Where does indieauth fit relative to that?
nolith[m], [chrisaldrich], Salt[m] and aviraldg joined the channel
# petermolnar funny fkery I recently ran into: my ISP doesn't seem to support ipv6 from/for my home. I enabled nginx on ipv6, and suddenly any service that runs on the google cloud started 404-ing on any resource on my site. How the hell does one debug this?
jamietanna[m] joined the channel
samwilson joined the channel
jalcine[m] and khimaros[m] joined the channel
# [Raphael_Luckom] neat. My understanding of oauth is that the service would need to decide to support indieauth as an identity source. So the idea is that indieauth could be like a distributed oauth, but it's most relevant to shared web apps like the wiki (possibly not my web site, unless I wanted a private section or something)
fwe joined the channel
# [Raphael_Luckom] cool thanks! seems like an interesting capability for down the road. I'm trying to pick the things to work on _first_
# [Raphael_Luckom] I think I'm going to start with webmentions and see how far it gets me 🙂
Pete[m], mayakate[m], nekr0z, smacko[m], fredcy_, [Ian_Forrester], Gyuri, JK_na, [scottgruber], [Simon_Willison], [jeremycherfas], schmudde, h0p3, DavidDylanThomas, maxwelljoslyn, [tonz], nickodd and [kimberlyhirsh] joined the channel
# [jgmac1106] I didn't realize vlc had a ripper built in...wow...and that is my daily driver as a media player
# petermolnar vlc is nearly as flexible as ffmpeg, but it has a gui
[schmarty], [tipoqueno] and Asher1 joined the channel
# [Simon_Willison] ↩️ We’ve been using Handbrake successfully
# [Simon_Willison] ↩️ oh but that was on Windows, haven’t run it on Mac
# 
[jeremycherfas] Glad to be of service. And it rips to flac, if that’s your thing.
[tantek], [hibs], tracydurnell, tomasparks, [Sue_Hanen], [calumryan], jenna, [KevinMarks] and [Ana_Rodrigues] joined the channel
# [jgmac1106] Pages to me is such a better design tool than Word
# [jgmac1106] Build me away to save games on archive.org on my website or anywhere
# [jgmac1106] my kids just can never finish Carmen San Diego...something happens before they finish
# [jgmac1106] can't get the javascript to work (I think) as I play with hibs variable font stuff: https://jgregorymcverry.com/indiewebpoetry
[manton] and miklb_ joined the channel
# 
Zegnat GWG, I have just pushed my new IndieAuth code to GitHub to try and pressure myself into finishing the code this weekend. So in case you are interested in what I am doing: https://github.com/Zegnat/php-mindee
GWG and [jeremycherfas] joined the channel
# [jgmac1106] ↩️ yeah might just break down and get a windows laptop to help with streaming stuff.
carla-paloma, [scottgruber], carolline, [Sue_Hanen], [Simon_Willison], reed, [dmitshur], saunders and [tantek] joined the channel; nickodd left the channel
# oenone hey.. I'm trying to use webmentiond for webmentions, but I get 405 when I try to submit a mention manually using curl. does anybody have an idea why or how I can make it log more verbosely?
reed and fwe joined the channel
# Loqi webmentiond is a standalone webmention receiver using Go and SQLite with backend UI https://indieweb.org/webmentiond
[Raphael_Luckom] and ShinyCyril joined the channel
[KevinMarks] joined the channel
miklb and GWG_ joined the channel
# [Simon_Willison] Datasette has plugins that add all kinds of different output formats - there are plugins to export data as Atom, ICS and YAML - here’s the atom one: https://github.com/simonw/datasette-atom
# [Simon_Willison] If I was going to build an IndieWeb export plugin, what format should I target first? ActivityPub/ActivityStreams? Something else?
# [Simon_Willison] The key idea would be that you could define a SQL query against any shape of data you like which results in content exported in a specific format. Here’s the query that defines the Atom feed for my til.simonwillison.net site for example https://til.simonwillison.net/til/feed
# [Simon_Willison] Add `.atom` to that page to get the output from the plugin
[jgmac1106] joined the channel
# [Simon_Willison] Datasette will happily turn any SQL query into an HTML table
# [KevinMarks] And add h-feed and h-entry?
# [KevinMarks] You have special handlers for different column types, right?
# [Simon_Willison] oh that’s interesting - I could do a plugin that gives you HTML with microformats (as opposed to the default table)
# [Simon_Willison] this is a demo which runs ALL of the Datasette plugins in one place: https://latest-with-plugins.datasette.io/github/commits - see the bit that says “This data as json, yaml, json-preview, copyable, CSV”
# [KevinMarks] What do you do for permalinks in atom?
# [Simon_Willison] You define those in your SQL as the `atom_id` column https://til.simonwillison.net/til/feed
# [Simon_Willison] ```SELECT
# [Simon_Willison] 'tag:til.simonwillison.net,2020-04-30:' || path as atom_id,
# [Simon_Willison] title as atom_title,
# [Simon_Willison] 'https://til.simonwillison.net/til/til/' || path as atom_link,
# [Simon_Willison] created_utc as atom_updated,
# [Simon_Willison] html as atom_content_html,
# [Simon_Willison] 'Simon Willison' as atom_author_name,
# [Simon_Willison] FROM
# [Simon_Willison] 'https://simonwillison.net/' as atom_author_uri
# [Simon_Willison] til
# [Simon_Willison] order by
# [Simon_Willison] limit
# [Simon_Willison] created_utc desc
# [Simon_Willison] 15```
# [Simon_Willison] the ICS plugin works in a similar way:
# [Simon_Willison] ```select
# [Simon_Willison] start as event_dtstart,
# [Simon_Willison] title as event_name,
# [Simon_Willison] events
# [Simon_Willison] description as event_descriptionfrom
# [Simon_Willison] order by
# [Simon_Willison] start```
# [Simon_Willison] ```select
# [Simon_Willison] title as event_name,
# [Simon_Willison] from
# [Simon_Willison] start as event_dtstart,
# [Simon_Willison] description as event_description
# [Simon_Willison] events
# [Simon_Willison] order by start```
# [Simon_Willison] https://github.com/simonw/datasette-ics#usage
# [Simon_Willison] I’m nervous about outputting HTML directly because of XSS risks, but I have plugins which avoid that by running their output through Mozilla’s Bleach library
# [KevinMarks] Need to check if that wipes mf2 classes (a lot of cleansing apis remove rel values, for example)
# [KevinMarks] So the permalinks for TIL go to markdown files? Those get htmlified on the fly?
# [Simon_Willison] The https://til.simonwillison.net/ used to render markdown using a Python markdown renderer from a custom template tag, but that wasn’t covering all of the weird extensions in GFM - so I ended up having my build script pass the markdown through their rendering API instead
# [Simon_Willison] https://til.simonwillison.net/til/til has a `body` column with markdown and a `html` column with that markdown rendered
# [Simon_Willison] In that case I can trust the HTML because it’s from me, but I need to watch out for people writing SQL queries that generate HTML, e.g. https://til.simonwillison.net/til?sql=select+%27%3Cscript%3Ealert%28%22bad%22%29%3C%2Fscript%3E%27+as+html
# [KevinMarks] Markdown is one of the things that is lossy wrt classes and rels
# [Simon_Willison] I do have a fun Datasette plugin called datasette-json-html that lets you define SQL queries that return JSON which is then safely converted into HTML - I could potentially add an allow-list of CSS classes to that
# [Simon_Willison] https://github.com/simonw/datasette-json-html
# [Simon_Willison] https://datasette-json-html.datasette.io/demo?sql=select+%27%5B%0D%0A++++%7B%0D%0A++++++++%22href%22%3A+%22https%3A%2F%2Fsimonwillison.net%2F%22%2C%0D%0A++++++++%22label%22%3A+%22Simon+Willison%22%0D%0A++++%7D%2C%0D%0A++++%7B%0D%0A++++++++%22href%22%3A+%22https%3A%2F%2Fgithub.com%2Fsimonw%2Fdatasette%22%2C%0D%0A++++++++%22label%22%3A+%22Datasette%22%0D%0A++++%7D%0D%0A%5D%27
# [KevinMarks] I am intrigued by the datasette approach to pesos as I have years of this stuff too
leg joined the channel
# [Simon_Willison] I honestly think the most interesting bit of it is using SQLite files as the intermediary - everything else becomes so much easier once you can run SQL queries, so I’ve put a ton of effort into making it as easy as possible to get stuff into SQLite - https://sqlite-utils.readthedocs.io/
# [Simon_Willison] All of the Dogsheep tools are built on that library
# [KevinMarks] Yes, that's it. I have been wary of database tax, but not trying to coerce into a single schema is interesting. You don't update the tables, just replace them in effect?
# [Simon_Willison] Depends on the application - if I have all of the data available in another format (e.g. HealthKit zip file of XML) then I throw away the tables (or delete the entire .db file) and recreate it entirely
# [Simon_Willison] if I’m pulling from the Twitter or Swarm API I just pull the most recent X entries and update the table in place
# [Simon_Willison] but yeah, a great thing about SQLite is that since a database is just a file it costs nothing to create a new one, or throw away an old one. I mainly treat them like ephemeral caches
# petermolnar I have one for site search
# petermolnar built & gathered from text files, fts4, served with a few lines of PHP after that
# [Simon_Willison] Yeah full-text search is definitely an appreciated feature of SQLite - it ships with pretty decent search built-in!
# [Simon_Willison] It’s not quite Elasticsearch but it’s Good Enough for most of my projects
# [Simon_Willison] has porter stemming, BM25 ranking - and since it’s SQL you can combine it with other WHERE clauses for advanced filtering
# petermolnar well, unless one needs stemming in Hungarian, but that is a whole different problem
# [Simon_Willison] yeah, if you want that you need to write a custom stemmer (probably in C) 😕
# petermolnar snowball (?) could handle it, but it's been a while since I played with that
# [Simon_Willison] https://github.com/abiliojr/fts5-snowball perhaps
# [Simon_Willison] oh wow i hadn’t seen that before,
# [Simon_Willison] ```CREATE VIRTUAL TABLE books USING fts5(author, title, description, tokenize = 'snowball spanish');```
# [Simon_Willison] only 11 commits, not clear if it’s actively maintained and works against most recent SQLite though
[hibs], [calumryan], geoffo and schmudde joined the channel; jmac left the channel
# [Simon_Willison] OK so I think I’ve figured out indieauth.com - I can construct this URL: https://indieauth.com/auth?me=simonwillison.net&client_id=https://simonwillison.net/&redirect_uri=https://simonwillison.net/
# [Simon_Willison] I then auth with indieauth.com and it redirects me to `https://simonwillison.net?code=…`
# [Simon_Willison] Then I take that code and do this:
# [Simon_Willison] ```httpx.post("https://indieauth.com/auth", data=
{"code": code, "redirect_uri": "https://simonwillison.net/", "client_id": "https://simonwillison.net/"})```# [Simon_Willison] And if the HTTP status code is 200 I know that the code was valid!
# [Simon_Willison] useful starting point for me
# 
jamietanna[m] <[Simon_Willison] "useful starting point for me"> Great stuff!
Guest2 joined the channel
# [KevinMarks] try https://indielogin.com/ as aaron is trying to separate the protocol from the site
geoffo joined the channel