#dev 2023-04-01
2023-04-01 UTC
IWSlackGateway, geoffo, [schmarty], [snarfed], gxt__, angelo and [tantek] joined the channel
# 
@roytang New blog post: Webmention Spam https://roytang.net/p/21cnqkib/ (twitter.com/_/status/1642085711900983296)
# 
IWDiscordRelay <capjamesg#4492> I have seen the same thing too ^
# IWDiscordRelay <capjamesg#4492> Maybe it’s pingbacks?
# IWDiscordRelay <Lunarequest#0122> By any chance ia the website rumnycircle?
strugee joined the channel
# IWDiscordRelay <capjamesg#4492> I’m seeing it from numerous different site Lunarequest[d].
# IWDiscordRelay <capjamesg#4492> I wonder if there is a way to separate pingbacks from webmentions on webmention.io…
# IWDiscordRelay <Lunarequest#0122> Hmmm funky, i suspected it was a website called rummycircle since they do shit like that and sms bomb peoplr
# IWDiscordRelay <Lunarequest#0122> [edit] Hmmm funky, i suspected it was a website called rummycircle since they do shit like that and sms bomb people
[jamietanna] joined the channel
# 
[jamietanna] https://twitter.com/aakashg0/status/1641976906982498310 is fun for those of us using Bridgy Publish
# 
@aakashg0 3. Links hurt, unless you have enough engagement Generally external links get you marked as spam. Unless you have enough engagement. https://pbs.twimg.com/media/Fsl5SwFXoAAolQx.jpg (twitter.com/_/status/1641976906982498310)
[schmarty] joined the channel
[TMichelleMoore], geoffo, chenghiz_, [pfefferle], nertzy, [tw2113_Slack_], [KevinMarks] and [snarfed] joined the channel
# [snarfed] It's not clear whether that will even be possible. As far as we can tell, they plan to turn off the v1.1 API entirely, and v2 doesn't have media upload yet. https://twittercommunity.com/t/i-got-a-error-when-i-posted-statuses-update-by-a-new-free-access-level/188995/2?u=schnarfed
geoffo and jjuran_ joined the channel
[chrisbergr] joined the channel
geoffo joined the channel
geoffo joined the channel
# bkil Or you using some existing library or system perhaps such as Elasticsearch or is this full-custom?
# 
[KevinMarks] The basic idea is that you convert them into weighting factors and multiply them
# [KevinMarks] Making it clear waht is going on in results is harder.
# [KevinMarks] With technorati, we had post tables with different time horizons. We'd look in the "last 24 hours" one first, then the last week, then last 28 days, then full.
# [KevinMarks] This was based on sorting by date primarily, and having very variable in links.
# bkil You could also consider using a logarithmic time scale.
# bkil I'm just guessing. Haven't implemented my own search engine yet, but am thinking about it. https://en.wikipedia.org/wiki/Temporal_information_retrieval#Time-aware_retrieval/ranking_models_(T-RModels)
# bkil So basically as per the multiplication mentioned by [KevinMarks] , you may multiply the existing weight by the (inverse) of the logarithm of the age of the given document. What base you use for your logarithm is subject for experimentation of course.
# bkil But I also like the binned approximation above - would probably be less resource intensive as well.
# bkil I'd be delighted to see your source if it's public. I collect links to such implementations: https://github.com/bkil/freedom-fighters/blob/master/en/service/web-search.md#user-content-foss https://github.com/bkil/freedom-fighters/blob/master/en/server/backend-optional-portal-search.md#user-content-references
# 
capjamesg The config required is non-trivial if you want to run it locally, but here's the code where I get info from the vector index: https://github.com/capjamesg/llm-chatbot/blob/main/PromptManager.py#L67
# [KevinMarks] The binned idea works when you have a lot of variation in results for different queries - eg for Technorati we were returning links to an article, and an NYT article would have a lot of links on the day compared to a blog post from 2 days ago
IWDiscordRelay, mouse[d], IWDiscord, geoffo, IWSlackGateway and [tantek] joined the channel