#dev 2016-08-16

2016-08-16 UTC
KevinMarks, mindB, mindB2, KevinMarks_ and rascul joined the channel
#
loqi.me
created /rust (+146) "prompted by KartikPrabhu and dfn added by rascul"
(view diff)
#
loqi.me
edited /rust (+45) "/* See Also */ new section"
(view diff)
#
rascul
is here
AngeloGladding joined the channel
#
rascul
i got distracted reading about soveriegn identity
#
AngeloGladding
hey rascul, KevinMarks_
#
rascul
where did we leave off?
#
AngeloGladding
i wanted to abstract 63-69 out of download function so that other distro-specifics could be handled at the main level
#
rascul
oh, yeah
#
AngeloGladding
KevinMarks_ is there dynamic "source"ing going on in those files or what do you want me to be looking at?
#
AngeloGladding
rascul: my problem specifically was..
#
rascul
btw i don't think fbsd has wget installed by default, but it does ship with fetch
#
rascul
so that might go into the os specific stuff also
#
AngeloGladding
cool.. i imagine ammending this file until it is utterly grotesque -- that'll make me feel complete
#
AngeloGladding
once I've got a nice clean Python3 env I can get back to my purity
#
AngeloGladding
so i'll remember that
#
AngeloGladding
but my problem..
#
AngeloGladding
setting shatemplate
#
AngeloGladding
shatemplate="SHA1 ($filename) = $sha1sum"
#
AngeloGladding
setting that inside the function allows me to use the variable
#
AngeloGladding
setting it outside the function.. is there any hope?
#
AngeloGladding
the local variables $filename and $sha1sum
#
rascul
you want to set it in the function but also use it outside the function?
#
AngeloGladding
hmm..
#
AngeloGladding
i want to rip that conditional out of `download()`
#
AngeloGladding
i don't know really a) if it's possible or b) how
#
rascul
ahh, so that should already be set before download()
#
AngeloGladding
right
#
AngeloGladding
was the idea
#
rascul
ok, not difficult
#
KevinMarks_
No, more that I end up writing python scripts that call other python scripts and do their own directory walking rather than using shell for it
#
rascul
essentially, move that whole if block to near the top of the script, before it gets to anything important, take out the lines that specify local for those variables, and adjust function calls to not pass those
#
rascul
so you're basically making those global variables and running the steps to set them right at the beginning
#
rascul
KevinMarks_ is that python script just for recursively copying files around?
#
AngeloGladding
KevinMarks_: have you ever solved the "source" a Py2 env from Py3 script or vice versa?
#
AngeloGladding
oh ok.. no need for locals.. everything proceeds in a single execution thread anyway..
#
rascul
yep, in this case
#
AngeloGladding
let me read those links you gave me, set up a testing environment and rerun w/ the changes
#
rascul
don't do any of those changes yet though, just focus on the download stuff
#
AngeloGladding
ok
#
rascul
read them and implement that stuff (if you so desire) later, once you get the download stuff changed and working
#
rascul
don't want to do to many changes at once
#
AngeloGladding
yeah and i want to give you the quickest feedback
#
AngeloGladding
give me a couple of minutes
#
AngeloGladding
so are you saying turn `local urlbase=$1` to `urlbase=$1`?
#
AngeloGladding
maybe you can advise a better overall approach
#
AngeloGladding
to lines 38-90
#
AngeloGladding
just download, verify and unpack
#
AngeloGladding
i'm tempted to turn it into python but I've been down this path before..
#
AngeloGladding
i'm certain bash can handle it
#
rascul
urlbase can stay local, but local isn't really necessary here because there's not a global $urlbase
#
rascul
move lines 63-69 up to the top, line 15 or so
#
AngeloGladding
done
#
rascul
you might need to declare $shafunc and $shatemplate before that though
#
rascul
oh, looks like that's all you might need to do
#
AngeloGladding
hmm..
#
rascul
you can also set other os specific stuff in those if statements if necessary so they'll apply to the entire script
#
AngeloGladding
the if statements contain the variable names from the download() function though
#
AngeloGladding
so when those strings are set, outside of the context of download() they appear to evaluate to ""
#
rascul
yes but they'll be global now
#
AngeloGladding
so define them outside of the if statements?
#
rascul
they'll be set globally
#
rascul
inside the if will set them globally
#
rascul
that's what you wanted? or did i misunderstand?
#
rascul
btw you should "quote" all those variables on lines 85-89
#
AngeloGladding
so $shatemplate contains variables that are set inside the download() function
#
rascul
also every time you cd "$variable" it should be quoted
#
AngeloGladding
i'm having a hard time describing the problem
#
AngeloGladding
do you see what i'm talking about?
#
rascul
hold on, lemme make a few changes and show you so you can let me know if i'm thinking the same thing you are or not
#
AngeloGladding
k
#
rascul
i'm also kinda drunk so my human interpretation skills might be subpar right now
#
AngeloGladding
several beers in myself
#
AngeloGladding
conclude the conversation at any time
#
AngeloGladding
i've got some reading to do
#
AngeloGladding
on bash that is
#
AngeloGladding
do you see my problem though w/ $shatemplate?
#
AngeloGladding
that's the crux
#
AngeloGladding
$shafunc is fine because its static
#
AngeloGladding
$shatemplate is reliant upon the variables passed into download()
#
rascul
oh, i missed that
#
rascul
no problem, though
#
rascul
i'm adding a slight bit of spice for you to ponder over too, almost done
#
AngeloGladding
k cool
#
AngeloGladding
to state the obvious you can exit out after the first call to download, supply the script with six arguments and test away
#
rascul
i think i got it, lemme look and make sure i did it right
#
AngeloGladding
i'm still here
#
AngeloGladding
?
#
rascul
nevermind, ijust thought of something then discarded it
#
AngeloGladding
i see your changes
#
rascul
don't look at them
#
rascul
it's a trick
#
rascul
you don't need the $os stuff at all
#
AngeloGladding
haha
#
Loqi
nice
#
AngeloGladding
if "type -p sha1sum > /dev/null && shafunc=$(type -p sha1sum)" isn't a trick i don't know what is
#
AngeloGladding
VOODOO MAGIC!
#
rascul
ignore that
#
AngeloGladding
:)
#
rascul
it wasn't going to work for you
#
AngeloGladding
carry on
#
AngeloGladding
i searched the google for cross-platform shell scripting
#
AngeloGladding
didn't find a canonical resource
#
AngeloGladding
seems like this kind of thing would be solved on stackoverflow or s/t
#
AngeloGladding
or like a forum circa '97
#
AngeloGladding
totally n00b here lookin' for some guidance :)
#
rascul
check it now
#
AngeloGladding
wk
#
rascul
line 17 means you can do 'shafunc=sha1sum ./plant_sapling.sh' and skip the os check
#
rascul
you might not use it or care for it, but i mostly did it to show you that trick
#
rascul
the $shatemplate actually could be set in the if statements up top but i didn't feel like trying to remember how to do that
#
rascul
and it could be potentially unsafe
#
AngeloGladding
quite the situation i've backed myself into here..
#
AngeloGladding
i guess.. fuck it..?
#
rascul
duck it
#
AngeloGladding
a bit of verbosity won't kill a bootstrap script
#
AngeloGladding
lol
KevinMarks joined the channel
#
AngeloGladding
but can you describe your line 17 magic?
#
rascul
also because of the if at line 26 it will exit instead of trying magic it can't do
#
rascul
it's the first one
#
rascul
${parameter:-word}
#
rascul
If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted.
#
rascul
it's more useful for setting default values and letting them be overridden from the environment
#
AngeloGladding
so what is the word in this case?
#
rascul
the word here is nothing
#
rascul
it's making sure the variable is initialized
#
AngeloGladding
interesting
#
AngeloGladding
alright
#
rascul
so that way the if on the next line sees that it's empty and figures it out, or sees that it's set and skips the os check because it's not necessary
#
rascul
but then if it's not set right, the stuff in download() will go bonkers though
#
AngeloGladding
real quick
#
AngeloGladding
did you pick up on my question earlier about "source"ing a virtual env from within a python script?
#
rascul
feel free to use it or not, i mostly just put it there to show you that trick
#
rascul
no, i missed it
#
rascul
oh, virtualenv
#
AngeloGladding
i'm learning everything about bash at this point
#
rascul
i don't use that crap
#
AngeloGladding
oh ok
#
AngeloGladding
haha
#
rascul
100% useless for me
#
AngeloGladding
no python?
#
rascul
i python often
#
AngeloGladding
no venv?
#
rascul
i setup single purpose users for stuff that people like to use virtualenv for
#
rascul
clean environment plus priviledge separation
#
rascul
then i can pip install --user whatever and not care
#
AngeloGladding
ahh.. --user flag
#
rascul
no need for hacky stuff when it's so simple to begin with
#
AngeloGladding
haven't used it but heard of it
#
AngeloGladding
kind of a pseudo-venv
#
AngeloGladding
alright
#
rascul
no chance of env overlap or anything crazy
#
AngeloGladding
right
#
AngeloGladding
envs are kinda hard to manage
#
rascul
because it's a spearate user specifically and only for that app
#
rascul
then with uwsgi you can pass user and group
#
rascul
to run the app as
#
AngeloGladding
interesting..
#
AngeloGladding
how about a VPS setup
#
rascul
that's how i do it, exactly
#
AngeloGladding
valid
#
AngeloGladding
?
#
AngeloGladding
cool
#
AngeloGladding
i'll check that out as well
#
AngeloGladding
i'll read about those bash changes you've recommended
#
rascul
no need for that fancy crazy hacky stuff to do what linux has done since the beginning
#
rascul
for dev stuff it's much more useful because you probably don't want to play with different users all the time
#
rascul
but that's not something i've ever had to concern myself with for my stuff
#
AngeloGladding
but for deployment.. so far all hosts provide multiple users..
#
AngeloGladding
that's probably the best takeaway here
#
AngeloGladding
thnx
#
rascul
then you can use the os permission and ownership concept and acls and stuff to tune priviledges and whatnot as needed
#
rascul
and at that point it's all simple, basic stuff that every admin knows and loves
#
AngeloGladding
i'm going to ponder the consequences of each sober
#
AngeloGladding
and i'll keep you apprised
#
AngeloGladding
:)
#
rascul
i can give examples, i host python and php and whatever the same exact way
#
AngeloGladding
could simplify things a bit
#
AngeloGladding
so what about system packages
#
rascul
create new user, install app into home, throw a new config file in nginx and uwsgi, live happily ever after
#
AngeloGladding
?
#
rascul
what about them?
#
AngeloGladding
do they carr over?
#
rascul
like system installed python stuff?
#
AngeloGladding
yeah
#
rascul
use pip install --user
#
rascul
and install the stuff in requirements.txt with the versions
#
rascul
and pip will ensure you have the appropriate version installed
#
rascul
the stuff in ~ will be picked up first
#
rascul
based on $PYTHONPATH
#
AngeloGladding
damn..
#
AngeloGladding
the options
#
rascul
if you have the correct version already installed system wide, pip won't bother installing it in ~
#
AngeloGladding
i feel like that'd be great on a single-user VPS but then i think of my dev machine
#
AngeloGladding
i'd love the installer to work in both contexts
#
AngeloGladding
...
#
rascul
that's doable
#
AngeloGladding
and shit
#
rascul
just need a --venv and a --user options, and require one of the two
#
AngeloGladding
wait
#
rascul
or something of the sort
#
AngeloGladding
the whole point of needing to source was this:
#
AngeloGladding
my setup requires supervisord which requires 2.7
#
AngeloGladding
and my code is written in 3.x
#
AngeloGladding
is that compatibile
#
AngeloGladding
bile..
#
AngeloGladding
that's how py's venvs are making me feel
#
AngeloGladding
:)
#
rascul
what kind of application is this?
#
AngeloGladding
so the script is downloading nginx, redis, creating a py27 for supervisor to manage nginx, redis, a uwsgi app, a background task manager and a crawler, .. more to come
#
AngeloGladding
yes
#
rascul
what does it need supervisor for?
#
AngeloGladding
manage all the things
#
AngeloGladding
and more to come
#
AngeloGladding
damnit loqi not now
#
AngeloGladding
:)
#
rascul
heh Loqi
#
rascul
uwsgi can run as a service and handle the process management type stuff
#
rascul
might be you don't need supervisor at all
#
AngeloGladding
so if you look at line 222 of https://gist.github.com/angelogladding/c01eb1a1707804864a6235117e3e886f you'll see I've got multiple programs i'd like to manage
#
AngeloGladding
swarm is a task manager.. sequentially fulfilling bg tasks..
#
AngeloGladding
kaleidoscope is a semantic crawler
#
AngeloGladding
canopy is a uwsgi app
#
AngeloGladding
i intend to have at least a bittorrent app as well
#
AngeloGladding
*definitely* going to have more manageable services
#
AngeloGladding
so while i hate that i need a second env to accomodate supervisor
#
rascul
nginx and redis are probably already setup to be capable of running as system services, managed by init
#
rascul
same with uwsgi, which can manage canopy
#
AngeloGladding
is that even a thing for VPS?
#
rascul
vps doesn't matter in this context
#
rascul
it's just linux
#
rascul
or, freebsd
#
AngeloGladding
init is a weak spot of mine
#
rascul
freebsd is a bit different, and i don't currently have any freebsd boxes
#
rascul
but what linux distro?
#
AngeloGladding
debian is most familiar
#
rascul
same conce3pts apply on fbsd though, they just do it a little differently
#
rascul
debian is easy peasy
#
rascul
for uwsgi, just put the config in /etc/uwsgi-emporer/vassals
#
AngeloGladding
well i suppose the more important one is my host
#
rascul
then probably systemctl enable uwsgi-emporer; systemctl start uwsgi-emporer
#
AngeloGladding
webfaction: CentOS
#
AngeloGladding
that requires su?
#
rascul
i don't know much about centos, but probably almost the exact same thing applies
#
rascul
well, sudo or su
#
AngeloGladding
vps don't got no r00t
#
AngeloGladding
:)
#
rascul
sure it does
#
AngeloGladding
er?
#
rascul
'sudo -i' gives you root
#
AngeloGladding
asteria is not in the sudoers file. This incident will be reported.
#
AngeloGladding
""
#
AngeloGladding
lol
#
rascul
do you own this vps?
#
AngeloGladding
wait
#
AngeloGladding
sorry
#
AngeloGladding
not vps?
#
rascul
doesn't matter if it's a vps or not
#
AngeloGladding
i'm targeting the lowest cost hosting
#
rascul
it's just linux
#
rascul
vps though is the lowest cost hosting you could target
#
rascul
this doesn't appear to be something you could do generally with shared hosting
#
AngeloGladding
https://www.webfaction.com/ scroll down -- shared @ $10/mo
#
rascul
digitalocean is $5/month for a vps
tantek joined the channel
#
AngeloGladding
and that'll give me root?
#
rascul
then you can do essentially whatever you want
#
AngeloGladding
now /that's/ the takeaway
#
AngeloGladding
of course...
#
rascul
i can give you referral for digitalocean, you get $10 credit
#
AngeloGladding
100% of the other homebrew attendees are on 1&1 and GoDaddy..
#
rascul
godaddy is disgusting
#
AngeloGladding
homebrew club LA that is
#
rascul
i have no comment on 1&1
#
AngeloGladding
i know....
#
rascul
what is digitalocean?
#
Loqi
DigitalOcean is a virtual private server provider targeted towards developers and offers low cost cloud servers in data centers across the world https://indieweb.org/DigitalOcean
#
rascul
there we go
#
AngeloGladding
damn
#
AngeloGladding
keep it simple
#
AngeloGladding
or do it right
#
AngeloGladding
supervise everything out of the home dir and support as many providers as possible
#
AngeloGladding
...
#
AngeloGladding
now i see i have realistic options
#
AngeloGladding
thought root-level hosting was out of the price range
#
AngeloGladding
sheesh.. gettin' tired
#
AngeloGladding
i'll look into my options tomorrow
#
AngeloGladding
just want to get my code live on all the hosts
#
AngeloGladding
just want to get my code live on all the things
#
AngeloGladding
don't leave me hangin' loqi
#
AngeloGladding
...
#
AngeloGladding
you rascal
#
AngeloGladding
oh wow
#
AngeloGladding
haha
#
Loqi
awesome
#
AngeloGladding
lol
#
rascul
i'm not a rascal
#
AngeloGladding
don't even know where to begin
#
AngeloGladding
called loqi a rascal for not meming my last statement
#
AngeloGladding
clearly i've been brainwashed by your username
#
AngeloGladding
and am delerious at this late hour
#
AngeloGladding
+ too much bash
#
AngeloGladding
:)
#
rascul
my name is ray schulz
#
rascul
take out every third letter and the space
#
AngeloGladding
nice
#
AngeloGladding
and i'm angelo
#
AngeloGladding
nice to meet you ray
#
rascul
do i have to take out any letters?
#
AngeloGladding
verbatim.
#
rascul
such boring
#
AngeloGladding
such
#
AngeloGladding
get rascul.io live and i'll add it to my crawl
#
rascul
no way
#
rascul
costs too much for .io when i can get domains for $1
#
AngeloGladding
but your github
#
rascul
https://rascul.xyz now but nothing there yet
#
rascul
my github is stupid
#
AngeloGladding
lol
#
rascul
i don't even know what i have there
#
AngeloGladding
static html derived from a rust generator
#
AngeloGladding
:)
#
rascul
oh, i think i just put that there
#
AngeloGladding
and we come full circle
#
rascul
i don't like github because i don't host it
#
AngeloGladding
wow man
#
AngeloGladding
check my github in the next couple of days
#
AngeloGladding
i feel the same way
#
rascul
remind me tomorrow i'm sleepy
#
rascul
what is gogs?
#
Loqi
Gogs is a self-hosted Git server with additional features such as issue management https://indieweb.org/Gogs
#
AngeloGladding
`canopy-software` should obviate github
#
AngeloGladding
alright
#
AngeloGladding
i'll see you in the chat no doubt
#
AngeloGladding
gnight
#
rascul
good night
#
Loqi
goodnight!
#
AngeloGladding
RASCAL!
#
rascul
snarls
KevinMarks_, KevinMarks, tantek, loicm_ and AngeloGladding joined the channel
rMdes joined the channel
#
@RikMende
@botwikidotorg @Veronica Great! are you implementing #webmentions on the botwiki wiki ?
(twitter.com/_/status/765497372415844352)
mindB, doesntgolf, cmal and rMdes_ joined the channel
rMdes_, singpolyma, ChrisAldrich, KevinMarks, mindB, gRegorLove and rMdes joined the channel
#
aaronpk
omg i just realized i have a very short path to implement my collection posts again
cmal joined the channel
#
upon2020.com
edited /UBOS (+360) "Updated for current release"
(view diff)
rMdes_ joined the channel
#
gregorlove.com
edited /Events (+32) "/* August */ LA"
(view diff)
#
gRegorLove
ChrisAldrich: ^ that should make LA show up on /this-week for 8/24 now
#
gRegorLove
aaronpk: this-week uses the locations listed on /events, right, not the p-location on the event permalink?
#
ChrisAldrich
Thanks gRegorLove, it'd have taken me ages to figure that out
doesntgolf joined the channel
#
gRegorLove
It's confusing. It looks at the event permalink to get the photo, so it seems it would look there for locations too.
#
gRegorLove
So clarifying with Aaron to be sure
chrisaldrich1 and KevinMarks joined the channel
#
aaronpk
correct, it only gets the photo off of the permalink. everything else is from the list
#
aaronpk
would probably be a good idea to use the permalink info for that too, but hey that's what happens when this code slowly grows
#
aaronpk
wow i forgot i had made that repo
#
aaronpk
one commit from 2 years ago, and that's nothing like the code that's running now :P
#
gregorlove.com
edited /Events (+16) "archive past events"
(view diff)
#
gregorlove.com
created /events/2016 (+298) "stub"
(view diff)
rascul joined the channel
#
gregorlove.com
edited /events/2016 (+6365) "/* 2016 */ archive Jan/Feb"
(view diff)
#
gregorlove.com
edited /Events () "(-6365) /* Recent */ archived Jan/Feb"
(view diff)
#
aaronpk
oookay i just updated this with the code that's actually used to generate the newsletters https://github.com/indieweb/this-week
#
GWG
Reading Vouch
#
GWG
I am having some structural problems with Vouch.
AngeloGladding joined the channel
#
aaronpk
structural?
#
GWG
aaronpk, how I can implement it in WordPress effectively.
#
GWG
In the meantime, WordPress 4
#
GWG
.6 just dropped
#
GWG
It has one small change to help Webmentions
#
gRegorLove
GWG: I think ben_thatmustbeme's method is a good guide. I followed that, at least.
#
GWG
I have three more that I hope to get into 4.7
#
gRegorLove
record referrers, async verify they link to you, store verified ones as possible vouches.
#
GWG
Gregorlove, you know WordPress structure, any ideas? I am giving some serious thought to writing in preliminary support in my experimental plugin.
#
GWG
I am stalled on my other project so I think I could use a switch.
#
GWG
Of course, now I have to check all my plugins for 4.6 compatibility issues.
#
aaronpk
speaking of upgrades, Laravel 5.3 is almost released, so i'm going to have to upgrade my site to that soon
#
gRegorLove
I would use WP cron to process referrers regularly
#
GWG
I am worried about race conditions
#
gRegorLove
Example?
#
gRegorLove
(I'm presuming you mean sending and receiving Vouch, btw. Referer stuff is needed only for sending.)
#
gRegorLove
Receiving and verifying might be an easier first step
#
GWG
I worry about expensive queries in wp cron.
#
gRegorLove
They're not expensive
rMdes joined the channel
#
gRegorLove
I process only one row at a time, but that's mainly because of the latency of getting the URL's HTML and verifying it links back.
#
gRegorLove
Which reminds me I need to do some heuristics about huge pages. Ran into a problem with it trying to verify a huuuuge page (which was spam)
#
chegalabonga.com
created /User:Chegalabonga.com (+981) "Created page with "Hi guys and gals! My name is David Peach and I am in the process of moving posts from various places to my new permanent domain, https://chegalabonga.com Chegalabonga is a made...""
(view diff)
#
aaronpk
always a little terrifying updating dependencies. hopefully all of these changes are backwards compatible.
chrisaldrich_, KevinMarks and singpolyma joined the channel
#
aaronpk
is attempting to add search to his site!
#
rascul.xyz
created /User:Rascul.xyz (+297) "init"
(view diff)
#
rascul.xyz
edited /IRC_People (-10) "update me"
(view diff)
#
rascul.xyz
edited /User:Rascul.io () "(-683) redirect to rascul.xyz because rascul.io is no more"
(view diff)
#
rascul.xyz
edited /HTTPS (-35) "change rascul.io to rascul.xyz"
(view diff)
#
rascul
how do i make {{rascul}} on the wiki go to User:Rascul.xyz instead of User:Rascul.io?
#
rascul
ie the https page
#
aaronpk
what is sparkline?
#
Loqi
A sparkline is a small, simple, data-dense graphic with typographic resolution that is shown inline and in-context with relevant text https://indieweb.org/sparkline
#
rascul
ahh it's Template:rascul thanks
#
rascul.xyz
edited /Template:rascul (-2) "change rascul.io to rascul.xyz"
(view diff)
KevinMarks joined the channel
#
aaronpk
ahhhh exciting! search is working!
#
aaronpk
now i want to add a map interface so i can search by location!
#
rascul
oh, minor edits don't get shown in here
#
rascul
they're secret!
KevinMarks and ChrisAldrich joined the channel
#
rascul.xyz
edited /HTTPS (+6) "/* Obtain */ let's encrypt is no longer beta, client is now from eff and called certbot"
(view diff)
#
aaronpk
rascul++
#
Loqi
rascul has 36 karma (1 in this channel)
#
rascul.xyz
edited /HTTPS (+119) "/* Obtain */ add wosign"
(view diff)
#
rascul
hrm much of the let's encrypt section needs changing also
#
rascul
probably isn't important to include instructions anymore since they're nicer to read from https://certbot.eff.org
#
bear
yea, letsencrypt section hasn't been touched since alpha
#
bear
rascul++ for keeping security info fresh
#
Loqi
rascul has 37 karma (2 in this channel)
#
aaronpk
would you assume search results on my site would be in reverse chronological rather than by highest relevancy?
#
rascul
what would determine relevancy?
#
aaronpk
search magic
#
rascul
i think maybe relevancy, but an option to sort chronologically might also be nice
#
aaronpk
i think i'mg oing to default to reverse chronological until i come up with a use case for sorting by relevancy
#
aaronpk
takes a while to add all my posts to the index!
#
rascul
i'm going to look more at the https page later, got some things to do for a bit
KevinMarks and KevinMarks_ joined the channel