man i just had a series of extremely good ideas* that are very simple and very implementable for #sciop that i think will cause an absolutely disgusting amount of (good, intrinsically deduplicating, actually decrease server load by creating a supporting swarm of peers) public data scraping to happen and basically lower the barrier to scouting endangered datasets to zero

*if you received the message flood of me having them you are not allowed to tell people if they are actually bad

Hackathon: Data Under Threat / Data Rescueing (Aug 7) in #München

The LMU Open Science Center (@lmu_osc) runs a hackathon to support the #SciOp #SafeguardingResearch initiative: Rescuing research data that is deleted by the Trump administration.

Bonus: @lavaeolus will give an ignition talk!

Thursday, 2025-08-07, 16 – 19 (only in-person)
Details and signup: https://github.com/lmu-osc/safeguar.de-hackathon

Become a data rescuer by turning your own laptop into a Research Data Rescue Node, scraping at-risk data sets, and breathing new life into your old HDD as part of a global, decentralised network.

#LMUMünchen #OpenScience #OpenData #DataRescue
CC @SafeguardingResearch @bitsUndBaeumeAuxMuc

I revived an old HDD with a #RaspberryPi Zero W 2 for #DataRescue:ing:

It runs ...

(a) a Bittorrent client that seeds at-risk data sets from the #SciOp database
(b) the `sciop-scraper` script to get new datasets into the swarm

Setup instructions for the Pi Zero: https://codeberg.org/nicebread/HiveSeed/src/branch/main/L1-RDRN_RPi.md

Setup instructions for `sciop-scrape` (on macOS & RPi): https://codeberg.org/nicebread/HiveSeed/src/branch/main/L1-sciop-scrape.md

Let me know if the instructions work for you; happy to collaborate on the manual.

#WasFehlt (?): Ein Torrent-Client für #SciOp und ähnliche Projekte, bei dem ich nicht festlegen muss, was ich seeden will, sondern der das automatisch entscheidet. Je nach dem, wo Kapazität gebraucht wird, und je nach dem, wie viel Speicherplatz ich ihm dafür zur Verfügung stelle.

Added a 10 Terabyte seeding node to the #SciOp #SafeguardingResearch swarm; focusing on large (> 1 TB) data sets with 0 or 1 seeders.

You may have heard that globalchange.gov and all the national reports on climate change have gone down.

We got em all on #sciop, a webrip and all the PDFs extracted: https://sciop.net/datasets/globalchange-gov-webrip

Edit: context - https://apnews.com/article/climate-change-national-assessment-nasa-white-house-057cec699caef90832d8b10f21a6ffe8

About US research data under threat and how everyone can contribute to saving it - @lavaeolus and me were interviewed in the TU Delft paper Delta: https://delta.tudelft.nl/en/article/saving-academic-data-is-easier-than-you-think-and-you-can-do-it-too mostly regarding our parts in the Safeguarding Research and Culture initiative at https://safeguar.de
#TUDelft #SafeguardingResearch #SciOp

check this out if you want to help preserve the archive of "most local newspapers through most of US history" that had its funding pulled, even if you only have a couple dozen gigabytes to spare, you can
a) make an account on https://sciop.net/ ,
b) run a qbittorrent instance, go to preferences>web ui and click enable,

and just do this

python -m pip install sciop-scraping
sciop-cli login
sciop-cli client login
sciop-scrape chronicling-america --next

and that's all.

if you have spare storage, you can sort by seeders, ascending, and start from there. or subscribe to the rss feed and auto-download it.

this is an archive funded by the library of congress (threatened) and the national endowment for the humanities (actively being eliminated). the alternative is that an enormous amount of US history that doesn't percolate into history books is owned and operated by lexisnexis and other for-profit data brokers.

this is the first run of some tooling to lower the bar for participatory scraping - at the moment, the archive is still online, and the scraper will automatically embed a webseed URL in the created torrent. so even if you don't have space to seed, you can scrape the data, upload the torrent, and make it possible for waiting peers to become mirrors