Jason2357

@Jason2357@lemmy.ca

• ⁨Comment⁩ on ⁨Just the way we likes it.⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:

  I usually treat them by using an extremely well established library where someone else has spent the requisite years crying over every stupid edge case of csv reading. Rolling your own csv reader is a bit like encryption. Until someone hands you a file that rejects all sanity and you start fking with regex. Lol.

• ⁨Comment⁩ on ⁨Just the way we likes it.⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:

  The delimiter isn’t really the issue. Its that there are lots and lots of weird edge cases that break reading csvs. If you use commas, at minimum, you need to escape commas in the data, or quote strings that might contain commas… But now you have to deal with the possibility of a quote character or your escape character in the data.

  Then you have the fact that csvs can be written with so many different character encodings, mangling special characters where they occur.

  Aaand then you have all the issues that come with lack of metadata - good formats will at least tell you the type of data in each column so you dont have to guess them.

  Lets see, its also really annoying to include any binary data in a csv, theres no redundancy or parity checks to catch currupted data, and they arent compressed so you need to tack on compression if you want efficient storage, but that means you always have to read the whole csv file for any task.

  Oh, that brings me to the joys of modern columnar formats where you can read selected columns super fast without reading the whole file.

  Oh god, I really kept going there. Sorry. Its been a year.

• ⁨Comment⁩ on ⁨Just the way we likes it.⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  It was some sort of weird database frontend the contractor used. It was very limited.

• ⁨Comment⁩ on ⁨Just the way we likes it.⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  P.s. in the above quagmire, the only solution is choose to keep only the most important un-clean column per csv, and make it the last column in the file so you have predictable columns. If you need more, then write separate csvs. Computers are stupid.

• ⁨Comment⁩ on ⁨Just the way we likes it.⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  God I hate csv with the fire of a thousand suns.

  Contractors never seem to know how to write them correctly. Last year, one even provided “csv”s that were just Oracle error messages. lol. Another told me their system could not quote string columns nor escape commas or use anything but commas as their separator, so there were unpredictable numbers of commas in the rows when the actual data contained commas. Total nightmare. And so much of my data has special character issues because somewhere in the pipeline a text encoding was wrong and there is exactly one mangled character in 5 million lines for me to find.

  Give me the data as closely to the source data as you can. If it is a database, then a database dump or access to a clone of your database is the best option by far. I don’t care how obscure your shit is, Ill do the conversion myself.

  For intermediate data, something like parquet or language specific formats like Rdata or pickle files. Maaaaybe very carefully created csv files for archival purposes, but even then, I think parquet is safe for the long haul nowadays.

• ⁨Comment⁩ on ⁨Scientific Exposure⁩ ⁨⁨4⁩ ⁨months⁩ ago⁩:

  Uuhh, beyond the fucked up publishing system, your advisor was a self destructive dick. It was his job to pay that. His lab and career benefit and hes the one that gets funding for research operations.

• ⁨Comment⁩ on ⁨Scientific Exposure⁩ ⁨⁨4⁩ ⁨months⁩ ago⁩:

  And a few years later rich folk start mass downloading the same databases to train LLM models using the exact same methods to sell access to those. No FBI.

• ⁨Comment⁩ on ⁨Just in time⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  People expanded to places with resources that they could live in, or bring back home. There are no resources that we know of in space that are not more easily accessed on Earth, and living out there would require a material investment from Earth that would be devastating.

  Most of the Earth is currently empty of humans, while space is colder than Antarctica, and less accessible than both the top of Everest and the bottom of the Mariana trench. You could build a city in any of those 3 places easier than even low-earth-orbit and any other celestial body would be thousands of times harder still.

• ⁨Comment⁩ on ⁨Just in time⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  There’s no job from those times you couldn’t do today while literally living better than they did. Quit your job, give away everything you own and go live in a tent in the woods harvesting mushrooms: Your life would still be better than theirs because you would still have access to some emergency healthcare, foodbanks when you are starving, and be protected from marauding pillagers.

• ⁨Comment⁩ on ⁨Just in time⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  Even being a king of that time would be a brutal life in a lot of ways. Death all around and a piss-poor chance of surviving any given year with every bit of your body intact and functional, extremely limited dietary variety, and the smell. Oh god the smell.

• ⁨Comment⁩ on ⁨Just in time⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  Yeah, camping and campfires are nice. They are nice because they are temporary and by choice.

• ⁨Comment⁩ on ⁨Just in time⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  Yeah, this is very much glorifying the past, and probably the future. Medieval peasants would dream of sitting in a warm cubicle, well fed, while scrolling lemmy, if they could imagine it. Space colonization is probably impossible too.

• ⁨Comment⁩ on ⁨Why aren't Linux based mobile OSes more popular?⁩ ⁨⁨5⁩ ⁨months⁩ ago⁩:

  A lot of technical answers, but consider a social driver: Linux users and developers are a lot more likely to prefer to do their computing on a “real” computer with a keyboard and large screen. Therefore, Linux as a desktop/laptop OS will always be significantly ahead of mobile offerings.

• ⁨Comment⁩ on ⁨Trump Makes It Very Clear They’re Going To Turn TikTok Into A Right Wing Propaganda Machine⁩ ⁨⁨6⁩ ⁨months⁩ ago⁩:

  That’s why this is so terrifying. Twitter at least had a politically engaged userbase, so people noticed the changes. Tiktok users get political content only incidentally, and has far fewer actual journalists. They are going to get manipulated beyond reason.

• ⁨Comment⁩ on ⁨Can you think of any now?⁩ ⁨⁨6⁩ ⁨months⁩ ago⁩:

  That, and most traditional dairy consuming European cultures never actually drank milk. They made cheese and butter, then poured the remainder in the pig trough to turn those calories into pork.

• ⁨Comment⁩ on ⁨Train your brain⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  The post isn’t a sound legal argument, but it is an ethical one.

• ⁨Comment⁩ on ⁨Bring out the trumpets and pour out the beer⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  It’s pretty open. Step 1 is make income tax as painful as possible, step 2 is get rid of it in favour of tarrifs and other regressive revenue generating fees. And MAGA was already convinced that progressive taxation helped people they want hurt.

• ⁨Comment⁩ on ⁨Bring out the trumpets and pour out the beer⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  It’s corporate socialism later, when the government bails them out.

• ⁨Comment⁩ on ⁨nooo my genderinos⁩ ⁨⁨7⁩ ⁨months⁩ ago⁩:

  Gender isn’t part of biology (as a social construct) but the complexity of sex absolutely is.

• ⁨Comment⁩ on ⁨Help.⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  This kind of thins is just moral panic. Funny moral panic, but still pointless. There’s always been a tiny fraction of the population that is completely out to lunch, and there always will be.

• ⁨Comment⁩ on ⁨[deleted]⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  Agreed. I smiles at the blunt: no need to talk about this, just go try this. Then I read the comments. Guess I should call my dad.

• ⁨Comment⁩ on ⁨Why did our friends stop posting on social media?⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  Yeah, that was it. You could only post so many interesting updates, but a handful of people could “like” stuff all night long.

• ⁨Comment⁩ on ⁨Why did our friends stop posting on social media?⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  The move between seeing “your brother in law took the kids to the zoo” to “your brother in law liked this trash article” was such a jarring transition.

• ⁨Comment⁩ on ⁨Why did our friends stop posting on social media?⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  I made a minimal account just to use Marketplace and then a few months later was permanently locked out of marketplace because some AI decided I was suspicious. It’s not with it.

• ⁨Comment⁩ on ⁨The Promised LAN⁩ ⁨⁨8⁩ ⁨months⁩ ago⁩:

  There are a bunch of p2p VPN systems out there. First step would be finding the one that works best for you and your friends. Nowadays, I would stick with something that is Wireguard underneath, as the older projects were a lot slower.

• ⁨Comment⁩ on ⁨I cooked some food cubes.⁩ ⁨⁨11⁩ ⁨months⁩ ago⁩:

  Make a big flat pancake of batter and use a cookie-cutter?