lingo.lol is one of the many independent Mastodon servers you can use to participate in the fediverse.
A place for linguists, philologists, and other lovers of languages.

Server stats:

66
active users

#compression

0 posts0 participants0 posts today

So I have hundreds of videos of ~1 minute recorded from my phone ~10 years ago, and they generally don’t have that great compression, nor they are stored in a modern and advanced video format.

For archiving purposes, I want to take advantage of my workstation’s mighty GPU to process them so that the quality is approximately the same, but the file size would be strongly reduced.

Nevertheless, compressing videos is terribly hard, and way more complex than compressing pictures, so I wouldn’t really know how to do this, what format to use, what codec, what bitrate, what parameters to keep an eye on, etc.

I don’t care if the compression takes a lot of time, I just want smaller but good looking videos.

Any tips? (Links to guides and tutorials are ok too)

Also, unfortunately I am forced to use Windows for this (don’t ask me why 🫠), but I know nothing about Windows because I hate it. Practical software suggestions are very much welcome, too!

#ffmpeg#help#askFedi

Brand new PEP by @emmatyping to add Zstandard to the standard library:
peps.python.org/pep-0784/

Will it make it in to 3.14 before the feature freeze on 2025-05-06? It'll be close but it's possible!

The PEP also suggests namespacing the other compression libraries lzma, bz2 and zlib, with a 10-year deprecation for the old names.

Join the discussion to give your support, suggestions or feedback:

discuss.python.org/t/pep-784-a

Python Enhancement Proposals (PEPs)PEP 784 – Adding Zstandard to the standard library | peps.python.orgZstandard is a widely adopted, mature, and highly efficient compression standard. This PEP proposes adding a new module to the Python standard library containing a Python wrapper around Meta’s zstd library, the default implementation. Additionally, to a...
#PEP#PEP784#zstd
#TIL about Sloot digital coding system - the inventor claimed that he could compress a movie to just 8 KB.

It got a lot of high profile investors but just a few days before Sloot sold his invention, he died of a heart attack.

A hidden hard drive was found in his demonstration devices. Posthumous analysis showed his code uses a known technique and that he probably faked his demonstrations to buy time thinking he could improve the algorithm beyond its mathematical limits.

#compression #computerscience #math #theranos #programming #algorithms

BOOK LAUNCH — How to make books more sustainable?

→ How to make books more sustainable? Inspired by the image compression on its solar-powered website, Low-tech Magazine squeezed the article catalog of their three-volume book series into just one book. Compressing the content — an editorial and design choice — produces a larger reduction in resource use than printing on recycled paper could ever do.

→ During this book launch event, we present our "Compressed Book Edition", followed by a public discussion on sustainability in the book publishing business. We also have a number of copies for sale during the event.
18:45 Doors open
19:00 Book launch
19:30 Q&A
19:45 Book sale / snacks & drinks / networking / low-tech showcase ;)

📌 Barcelona — Akasha HUB, Carrer de la Verneda 19 (El Clot)

📝 sign-up on meetup! (meetup.com/akashabarcelona/eve)
🗯 poster by our intern Hugo Lopez

Have you ever thought about how amazing modern #video #compression is?

When #QuickTime came out in 1991, the video codecs didn't save you much space, and were very primitive. You could see the (temporally) repeating blocks very obviously, and they didn't have a lot of finesse.

Cinepak was the first one that had some real compression muscle, then Intel Indeo and others.

I've got a 113-second, 720x1600 video that SHOULD take up 21GB of data raw, but easily compresses down to 16MB, and I can hardly tell the difference.

That's 1360:1 savings!

Does anyone know the compression algorithm that was used in Connectix RAM Doubler which was available for Classic MacOS in the 1990s – or, this would be even better, was the software ever reverse engineered or open sourced?

Since Mavericks (10.9), modern Mach-based OS X/macOS uses the WKdm algorithm for memory compression, so it would be interesting to see if this was already used back then.

en.wikipedia.org/wiki/Virtual_

en.wikipedia.orgVirtual memory compression - Wikipedia

”LLM’s are storages of millions of vector programs, that are formed by passive exposure to human output.”

No. There are so many things wrong in this!

-Not programs because they don’t execute anything
-If you give an #LLM a mathematically recursive task, it inferences in exact same time regardless of complexity
-It’s more like #compression than programs, really
-And ”passive exposure”??!! Wtf are these people smoking?!!

Tells more about the psychologies of people building #ai 😣

Continued thread

And compression is now super fast!
💻Performance on Mac M1:
✅𝐂𝐨𝐦𝐩𝐫𝐞𝐬𝐬𝐢𝐨𝐧: 7 GB/s
✅𝐃𝐞𝐜𝐨𝐦𝐩𝐫𝐞𝐬𝐬𝐢𝐨𝐧: 8 GB/s
Wait till multithreading happens on GPU and you only decompress on demand

#compression

#llms

#GPUComputing

#ai

𝐏𝐚𝐩𝐞𝐫: alphaxiv.org/abs/2411.05239

Staying in bed for health reasons basically the whole day, I found a #UVictoria #lecture by Bill Bird about data #compression, starting from the absolute basics (minimal programming and data structure knowledge is kind of required though), explaining the algorithms in detail.

youtube.com/playlist?list=PLU4

19 videos, about 1 to 2 hours each, about lossless algorithms like gzip and bzip2, but also image and video compression.

I'm 3 hours in and find it really interesting so far.

youtube.comBefore you continue to YouTube
Continued thread

An interesting side-note:

Because #sunoai and other #genai music models have been trained illegally from compressed music obtained without permission from streaming & online services, it turns out there’s an almost sure-fire method of detecting #ai music output as well:

Psychoacoustic compression artefacts

Because the models learn everything, they’ve not learned the full spectra of instruments, but rather what the brain doesn’t discard when hearing them.

Hexlish Alphabet for English, Constructed Languages and Cryptography: Automatic, Structural Compression with a Phonetic Hexadecimal Alphabet

DOI : https://doi.org/10.5281/zenodo.13139469

Hexlish is a legible, sixteen-letter alphabet for writing the English language and for encoding text as legible base 16 or compressed binary. Texts composed using the alphabet are automatically compressed by exactly fifty percent when converted from Hexlish characters into binary characters. Although technically lossy, this syntactic compression enables recovery of the correct English letters via syntactic reconstruction. The implementer can predict the size of the compressed binary file and the size of the text that will result from decompression. Generally it is intuitive to recognize English alphabet analogues to Hexlish words. This makes Hexlish a legible alternative to the standard hexadecimal alphabet.

@conlang@a.gup.pe @languagelovers@a.gup.pe @linguistics@a.gup.pe @academicchatter@a.gup.pe

#Hexlish #Conlang #Alphabets #English #Hexadecimal #Encoding #Cryptography #Ciphers #Crypto #Encryption #Compression #Papers #Preprints

I love playing around with #compression

In this case, it's all text-based data in csv and xml formats.

Size:

32,696,320 202411.tar
 4,384,020 202411.tar.bz2
 4,015,912 202411.tar.zst
 3,878,583 202411.tar.bz3
 3,730,416 202411.tar.xz

zstd was invoked using zstd --ultra -22
xz was invoked using xz -9e
bzip2 was invoked using bzip2 -9
bzip3 has no compression level options

Speed:

zstd    54.31user 0.25system 0:54.60elapsed 99%CPU
xz      53.80user 0.06system 0:53.93elapsed 99%CPU
bzip2    5.33user 0.01system 0:05.35elapsed 99%CPU
bzip3    3.98user 0.02system 0:04.01elapsed 99%CPU

Maximum memory usage (RSS):

zstd    706,312
xz      300,480
bzip3    75,996
bzip2     7,680

*RSS sampled up to ten times per second during execution of the commands in question

#bzip3 is freaking amazing, yo.

#DataCompression #bzip #bz3 #zstd #zst #zstandard #xz #lzma
#CouldaBeenABlost ;)