#dev 2021-06-11

2021-06-11 UTC
[fluffy], [tantek] and [jacky] joined the channel
#
[jacky]
lmfao right
#
[jacky]
like granted this is only a problem b/c of how they store information
#
[jacky]
a distributed journaled file system
#
[jacky]
it sounds clever but like <shrugs>
[schmarty], KartikPrabhu and gRegorLove_ joined the channel
capjamesg, reed, calebjasik, Abhas[m], batkin[m] and nekr0z joined the channel
#
petermolnar
did they mention their blu-ray based cold storage? 'cause that's quite hard to delete.
oodani, hendursa1, KartikPrabhu, [grantcodes], [Murray], IWSlackGateway3, gRegor, [KevinMarks], hendursaga, [jacky], [tw2113_Slack_] and [pfefferle] joined the channel
#
sknebel
keep encryption keys online, wipe those
#
sknebel
or add "shredder" as a location to the blueray-moving bot :D
alex_, [tantek], BinyaminGreen[m], gRegor, [pfefferle]1, alex__, aaronpk_, michael, mikeputnam1 and HelpMeUnderstand joined the channel
#
[tantek]
^ intentional barriers 😂
[snarfed], rrix and [KevinMarks] joined the channel
#
[snarfed]
[manton] just fyi https://help.micro.blog/ seems unhappy
#
[snarfed]
looks like its SSL cert expired a couple days ago
reed joined the channel
#
[jacky]
manton: if you've been curious about expanding beyond Hugo, I think this might help https://www.mikestreety.co.uk/blog/creating-an-11ty-collection-from-json-api/
KartikPrabhu and gRegor joined the channel
#
GWG
Out of curiosity, what do people think a good default expiry is for an IndieAuth token?
#
gRegor
I've been using 14 days in my dev
#
GWG
gRegor: I was going back and forth...
#
GWG
The amount will be changeable in my plan on the authorize screen, so I suppose that 14 as a default is fine
#
GWG
How does OAuth handle it? Can a client tell you something that indicates what token type it should give?
#
aaronpk
no you wouldn't want the client to have any control over that. usually that's set by policies on the authorization server, which might depend on the client type or the user or the scopes issued
[fluffy] joined the channel
#
GWG
aaronpk: I know you wouldn't want the client...but how do I know what the client type is?
#
gRegor
My server defaults to 14 days and lets you check a box to make it not expire. There's also a UI to expire individual tokens or all at once.
#
aaronpk
the only indication you have in indieauth is the client_id URL and what scopes it's requesting
#
GWG
I'm more thinking of knowing whether a client is unattended or attended. So Quill vs OwnYourSwarm
#
GWG
Whether a user controls the flow
#
aaronpk
yeah that would be nice. we don't have anything in the flow that would indicate that
#
aaronpk
you see this in some oauth servers where they request the scope "offline_access"
#
GWG
aaronpk: Maybe that's a scope proposal
[manton] joined the channel
#
[manton]
[snarfed] Oops, thanks.
#
GWG
In the meantime, will have to start somewhere
#
gRegor
What are you thinking you would do different for offline access clients?
#
sknebel
they cant as easily prompt the user for a new token
#
GWG
gRegor: Might change the text at the least when authorizing.
#
GWG
Those are the use case for refresh
#
aaronpk
another example would be letting those apps get long lived tokens but limtiing the token lifetime on apps like quill that only work when the user is actively using them anyway
alex11 and barnaby joined the channel
#
barnaby
I’m posting a video to my site rather than yt for the first time in forever, trying to figure out what settings will get me the best results in the least space
#
barnaby
got a bunch of transcoding jobs queued up in handbrake
#
barnaby
so far, nothing has been smaller than the original 1080p mov file which DaVinci Resolve gave me
#
barnaby
the 1080p webm ended up being 190MB vs 133MB for the .mov
#
barnaby
which is especially weird as I’m pretty sure the webm is 30fps and the (much smaller) original was 50fps
#
barnaby
heh, and the mp4 is 150MB
#
barnaby
[snarfed]: do you know which video formats successfully get POSSEd to twitter via bridgy publish?
#
barnaby
seems like webm codec via handbrake is consistently much larger than .h264/mp4
#
barnaby
so, for my 01:44 long video, the smallest I managed to get it with handbrake was as a 720p H.264 mp4 file, at 85MB
#
barnaby
webm was ~1.5x larger than H.264 at both 1080p and 720p
#
aaronpk
does handbrake use ffmpeg under the hood or is it its own thing?
#
barnaby
hmm no idea
#
barnaby
likely ffmpeg, as the UI definitely looks like a thin layer over a command line tool :P
#
barnaby
do you know of a particularly good ffmpeg option set for encoding web videos?
#
aaronpk
I don't remember, it's been a while since I touched any of that. I think actually most of the time i've posted videos on my site it's either from instagram or just whatever my phone records not transcoded
#
aaronpk
for long videos i've sliced them up for HLS
#
barnaby
85MB it is then, for the moment
#
barnaby
so much for the webm hype :/ unless I’m missing something obvious
#
barnaby
those were with the VP8 encoder, I’ll try one with VP9 and see if it makes any difference
#
[snarfed]
Bridgy Publish currently only supports mp4 videos w/Twitter right now. I think they required that at some point, but maybe they’ve relaxed it since?
#
[snarfed]
https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/uploading-media/media-best-practices (Video specifications and recommendations section) has lots of info but doesn’t have explicit formats
#
barnaby
I just tried posting one, seems to be taking a while, but it took 3:30 to upload to my site so I imagine it will take some time to upload to twitter
#
[snarfed]
via Bridgy? or directly?
#
barnaby
via bridgy, as an mp4 with a u-video class
#
barnaby
still hasn’t shown up
#
[snarfed]
looking
#
aaronpk
i wonder what's a good threshold for deciding to publish HLS instead of a single video file
#
barnaby
yikes, webm VP9 is 151MB vs VP8 123MB, at the same settings
#
barnaby
I guess that’s why VP8 is the default setting
#
barnaby
or compatibility, maybe
#
barnaby
aaronpk: whatever it is, I’m guessing my 1:45 long video of wild hamster clips isn’t quite there yet
#
aaronpk
at 80mb+ it might be!
#
aaronpk
cause otherwise someone visits that page on a phone and it downloads the whole thing
#
barnaby
I don’t have it set to auto play!
#
sknebel
browsers should be clever enough to not load all of it if not needed
#
aaronpk
one trick i've been meaning to look into more is providing an HLS playlist that points to byte offsets in a single video file
#
sknebel
fairly sure even on desktop they only load the start and then fetch more if it is played
#
barnaby
so at least on ff I have to explicitly start playing it, and it takes a few seconds before it has buffered enough to start
#
barnaby
so it’s definitely not loading anything before it’s asked to
#
[snarfed]
barnaby I’m not seeing any recent bridgy publish requests for that page. did you do it interactively?
#
barnaby
[snarfed]: nope, should have been triggered via webmention. I’ll try re-saving the post and see if that works
#
[snarfed]
I do see your link to https://brid.gy/publish/twitter , maybe I’m missing the request
#
barnaby
more likely my webmention endpoint tried something stupid like loading an 85MB file :/
#
barnaby
oh yeah looks like it’s an issue on my side. saving just fails for some reason. I’ll look into it after dinner
#
[snarfed]
feel free to try interactive too, on https://brid.gy/twitter/BarnabyWalters
#
petermolnar
be memory limits - eg php-fpm -, but the python ones could go completely rogue.
#
petermolnar
> more likely my webmention endpoint tried something stupid like loading an 85MB file :/ - I'm still extremely tempted to respond with a certain gzip to queries like obvious wordpress sniffing on my static site, but the malicious person in me wonders how many webmention receivers could deal with a gzip bomb without taking down the server behind them. Depending on the implementation, there should
#
petermolnar
it's a good thing that unlike in xml and yaml, there is no option to make a recursive json.
[tantek] joined the channel
#
GWG
petermolnar: I actually do a head on mine to check type and I limit size
#
barnaby
GWG: yeah pretty sure that’s what I’ll be implementing after I’m done cooking :D
#
barnaby
unless it turns out to be a different issue
#
barnaby
I might also just filter out URLs ending with common media extensions e.g. png mp3 mp4 etc from webmention processing altogether
#
barnaby
I originally thought it might be interesting to send them so that people could get notified if I hotlink their content, but I don’t think anyone does that
#
petermolnar
GWG: can I test your webmention endpoint?
#
barnaby
I *must* be doing something wrong
#
GWG
petermolnar: Try wpdev.gwg
#
GWG
wpdev.gwg.us if you want
#
GWG
It's my test site
#
barnaby
I just downloaded a video from twitter, it’s a 720p H.264 mp4
#
GWG
It breaks regularly
#
barnaby
5.6MB for 26 seconds, that’s 0.22MB per second. My video is 100 seconds long, so it shouldn’t be much more than about 22MB
#
barnaby
but it’s 4x that
#
petermolnar
GWG: returned a http 500
#
Loqi
curl has -1 karma in this channel over the last year (2 in all channels)
[fluffy] joined the channel
#
sknebel
curl++
#
Loqi
curl has 0 karma in this channel over the last year (3 in all channels)
#
petermolnar
but it was a wordpress http 500, so it looks fine
#
GWG
Was there a message?
#
petermolnar
Uncaught Error: Cannot use object of type WP_Error as array
#
petermolnar
one sec, I need to paste it
#
sknebel
barnaby: in handbrake you can set a target bitrate - set that to match what you want and see if the result looks good?
#
GWG
petermolnar: That's helpful. But it would have returned an error message due size.. just didn't surface it
#
GWG
I know how to fix that. I'll make a note
#
GWG
But I put a lot of safety measures into that code
#
GWG
Including refusing to download media mime types
#
petermolnar
::insert Shaggy's Are you challenging me?! picture::
[jacky], [schmarty], gRegor, hendursa1, [Ana_Rodrigues] and [tw2113_Slack_] joined the channel
#
barnaby
okay, the bit rate definitely explains the video size difference! my video is 6Mb/s, the twitter sample is more like 1, which explains the vast size difference
#
barnaby
thanks sknebel
#
barnaby
so now to queue up a bunch of renders at different bit rates and decide where to make the size/quality tradeoff
#
barnaby
then to document it all on /video so I don’t have to think about it in the future
#
aaronpk
document++
#
Loqi
document has 1 karma over the last year
#
barnaby
maybe I can even upload a handbrake preset file somewhere
#
sknebel
also always depends a bit on the video
#
sknebel
how much bitrate it usefully needs
#
sknebel
but some starting points would be helpful
#
aaronpk
6mb/s is a good target for high quality 1080 video
#
barnaby
sknebel: that’s why my render queue currently looks like this: https://waterpigs.co.uk/img/screenshot-2021-06-11-at-231459.png
hendursaga joined the channel
#
barnaby
that’s a lot of hamster videos
#
barnaby
two minutes of 720p 3Mb/s is a much more reasonable 41MB, starting to see a noticeable quality dip now
#
[tantek]
is that like high resolution hamsterdance or something?
#
[tantek]
oops thought this was #indieweb-chat for some reason 🐹
#
barnaby
is happy to spread his wild hamster obsession everywhere
#
[snarfed]
tantek++ hamster dance is a deep cut
#
Loqi
tantek has 18 karma in this channel over the last year (62 in all channels)
#
barnaby
anyone know how the “Constant quality: RF [x]” slider works in handbrake?
#
barnaby
what seem like small changes on it can result in vast file size and quality differences
#
sknebel
it uses some video quality estimation thing to evaluate how individual pieces of the video look after compression, and adjusts the bitrate for the pieces until they all hit a specific quality value
#
barnaby
ah, an adaptive bitrate thing, cool
#
sknebel
not the clearest text, at least for my sleepy brain, but some details that match what I remembered
#
barnaby
at least for this video, it looks like RF 20-35 is the range where the quality is still acceptable, but the file size starts dropping dramatically
#
barnaby
nice link, thanks sknebel++
#
Loqi
sknebel has 7 karma in this channel over the last year (32 in all channels)
[aciccarello] joined the channel
#
barnaby
looks like the (now 25MB) video file is still too big for twitter?
#
aaronpk
isn' there a maximum duration for twitter too?
#
[snarfed]
“Large file can not be finalized synchronously.” 🤔
#
barnaby
I looked it up, it’s 2:20 IIRC
#
barnaby
my video is 1:45
#
aaronpk
wow that's an error message
#
barnaby
I edited it specifically to be twitter-compatible ;)
#
aaronpk
oh gosh
#
sknebel
https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/api-reference/post-media-upload-finalize <- seems like for large videos you need to tell it to finalize the upload and then poll for that to finish? maybe thats not supported in bridgy?
#
[snarfed]
yup, bridgy already does INIT, APPEND…, FINALIZE
#
sknebel
right, but can it do the polling if finalize doesn't succeed synchronously?
#
[snarfed]
i don’t…know?
#
sknebel
that'd be my guess given the error message and the response examples in that doc link
#
[snarfed]
heh, lots of this info is news to me since i implemented twitter video upload many years ago
#
[snarfed]
er whee
#
[snarfed]
i really shouldn’t have to know this much about twitter’s internal video processing
#
[snarfed]
OK barnaby feel free to try again. I give it a 50/50 chance this time 😎
#
barnaby
thanks, I’ll give it another go!
#
barnaby
interesting
#
[snarfed]
yup, video upload seems ok, but when bridgy then tried to tweet with it, twitter replied “Not valid video”
#
[snarfed]
gotta run but I can look later. apologies for the trouble!
#
barnaby
no worries, thanks a lot for the live support!
#
barnaby
I’ll resist the temptation to post it manually, so we can test it again later :)
#
barnaby
hmm or maybe I’ll try posting it manually, to see if the twitter web UI complains about the video file
#
barnaby
that way we know if it’s just twitter not liking my particular hamster video file, or a bridgy issue
#
barnaby
huh well the twitter web video upload UI is somewhat lacking
#
barnaby
I posted it, and the form just greys out, with no progress or activity indicator
#
barnaby
okay, twitter web upload worked eventually, so it looks like it’s not an issue with that particular file
#
[tantek]
at some point I gave up on posting video because it got so involved
#
barnaby
yeah me too. not something I can motivate myself to do very often, but in the case of the hamsters, it made sense. It’s not something I’d put on youtube, but is something I want on my site
wagle joined the channel