Hacker News new | past | comments | ask | show | jobs | submit login
Modern CSV version 2 (moderncsv.com)
324 points by evanem on Aug 15, 2023 | hide | past | favorite | 155 comments
Hi, I'm Evan and I developed Modern CSV. Last year, the beta version of version 2 was posted here on Hacker News. As a follow-up, I'm letting you guys know that the beta period is over and Modern CSV 2 is now available.

Modern CSV is a tabular file editor/viewer for Windows, Mac, and Linux. I developed it out of frustration with how spreadsheet programs handle CSV files. Plain text editors, on the other job, do a poor job of handling columns. With Modern CSV, I attempt to combine the best of both worlds.

With version 2, you can expect to see an improved UI/UX, better performance, more useful features, updated documentation, and for Mac users, native Apple Silicon support.

If you haven't tried it yet, give it a shot and let me know what you think!




You should add that "Modern CSV is a tabular file editor/viewer for Windows, Mac, and Linux" line to the announcement page on https://www.moderncsv.com/modern-csv-2-is-now-available/ - for this kind of post it's always best to assume that people landing on the page don't know what the software is yet.

(It's easy to assume people will click the link from that page to https://www.moderncsv.com/ in order to find out more, but I'm not sure that's always a safe assumption!)


Yeah, the HN title lead me to believe this would be version to of a modern cab specification.

A bit surprised to see that a specification had improved UI before it dawned on me that it’s a CSV reading.


Haha, I had the same thought "Oh wow, I wonder what sorts of modern data storage techniques they managed to backport into simple CSV" And then.... disappoint. At least until I check out the software. It still sounds like the same column functions I get from notepad++ or VS:C


To expand upon this... the name is a problem.

"How can one make a _modern_ version of the CSV file format, let alone a second?"


The name only refers to the app's features and appearance. Well before I started development, I was looking for a CSV editor for my own use and everything I tried had a 90s look and feel. I figured just because the format is ancient doesn't mean the applications for it should be.


>The name only refers to the app's features and appearance

Perhaps, but it's very easy to confuse with it being about a modern version of the CSV standard.

In fact that would be probably the default interpretation.


Yeah, that is the first thing I thought.

Modern CSV Editor will probably be a much better name.


Yeah, and perhaps sticking the word together: ModernCSV Editor


CSVEdit? Hopefully not sued by reddit!


I think it’s a great name. Keep it. Even if it confuses some people at first, it’s the sort of confusion which will inspire many people — especially those who deal with text files all day and get pedantic about this stuff — to satisfy their incredulity and thus be exposed to your program.

This is perfect for a tool which most people, including those who would derive the most benefit from it, won’t even hypothesise the existence of.


This type of marketing by deception is absolute anathema to me.

People will "dig deeper" because you tricked then into thinking what you've made of something else, something that person knows they want. Bleurgh.

Yes, that's consistent with modern marketing: make people get your product even if they don't want it and/or don't need it. But, really, are you so desperate. You might save a 1000 hours by making it clear what the subject is; choosing to waste others time [which I'm not at all claiming the OP has done] is contemptible.


Definitely not my intent.


And it's obvious that it wasn't. Don't apologise. Ignore the critics. Any real potential customer will be far too distracted by the excitement of discovering a new thing which might make their life easier; they're not going to be lingering on the moment of cognitive dissonance. Anyone who does was never going to be your customer anyway.


> it’s the sort of confusion which will inspire many people — especially those who deal with text files all day and get pedantic about this stuff — to satisfy their incredulity and thus be exposed to your program.

Confusion doesn't inspire, it irritates. And to pedantic people, the satisfaction comes from being rightfully irritated.

You're basically advocating in defense of misleading/false advertising.


>it’s the sort of confusion which will inspire many people

...to skip it, because while they could use a CSV file editor, they don't care to read about yet another "modern CSV" format.

Confusion can only inspire if potential users pass from hearing about this, to the next level (of checking what it is about).

I, for one, almost didn't click the HN submission because of this reason.


Ignore them, ModernCSV is perfect and memorable. That's how I remembered it and found it.


Of course those who skipped it because of the name wouldn't be posting...

https://medium.com/@penguinpress/an-excerpt-from-how-not-to-...


Something playing on CSV could work - CSVlad or CSViper or CSVi


CSVi is quite clever. It immediately conjures the idea that its an editor for CSVs.

I assumed from the HN headline that Modern CSV version 2 was an iteration on some new CSV specification.


CSVi is cute and clever in exactly the way you don't want to be. It loses the entire audience of people who are Excel jockeys working with CSV files and maybe frustrated with the experience. Software engineers and system admins know what vi is. The rest of the world doesn't.


The worse is the people who do know what vi is, and assume that it's a vi plugin or worse, an editor using vi bindings, and ignore it.


You know that was my first though as well (new CSV version), but then I thought that a new version wouldn't make sense since (1) there is already a specification (the RFC), and (2) those who don't care about the current RFC won't care about a new spec, either.

CSV is one of those things that are everywhere because it is “obvious”. Then we have to deal with 100 interpretations of “obvious” decades later, since not enough people care about standardization.


It is clever but I can imagine the comments the OP would've gotten if posting "CSVi" and it didn't use vi-like keyboard shortcuts!


Spoiler: it's Parquet.


Parquet is great but I'd argue ndjson[1] is closer to "modern" CSV. That is, people who like to describe things as "modern" and "elegant" are likely to want JSON everywhere ;-)

[1]: Newline-delimited JSON, http://ndjson.org/


i agree with the nibling comment that this name works precisely because it provokes curiosity about its relevance.

i can't be the only dev who saw this headline and wondered whether some entity was pushing for a csv standardization (e.g. guarantees about encoding, escaping, etc.)


No no no, not CSVi, but CSVmacs!!! ;)


> How can one make a _modern_ version of the CSV file format

Check out WebCSV*:

<http://www.w3.org/TR/tabular-data-primer/>

* I'm aware that "WebCSV" isn't the real name, but it should be.


That sort of thing usually breeds curiosity and leads someone to dig in deeper about the product though.


I don’t really see the problem. Of course, CSV is more of a vague description of a family of pseudo-standards, rather than an actual standard. But Modern CSV seems like a fine name for an actual standard that takes inspiration from them.


That's exactly the problem.

This isn't about a name "for an actual standard that takes inspiration from them" but for a CSV editing software.


Just wanted to say I haven't used your product yet, but there are a couple things I really like about your site that led me to download it:

1. The first question I had was "How is this better than Excel?" The Google result for your site directly pulls up this very informative blog post, https://www.moderncsv.com/why-excel-sucks-and-modern-csv-is-... . Note my 2 cents is that I would more prominently link this, or at least have a more prominent section, from your home page.

2. Simple AF pricing. So rare these days. Kudos!


Thanks for linking that post. I would summarize this tool more as a log viewer and editor, not necessarily a CSV editor. I mean, this: "What if you’re running a script that writes to a CSV file and you want to see it updated?" I have never experienced in my life, and I had my fair share of CSV munging...


I would use that feature. We do data processing, and for testing I developed a version that runs locally on CSV files. It can take a few minutes to work through a large file, but it's stream processing so it starts writing almost immediately. Being able to load that output sooner than once it's done like Excel provides would be a nice improvement.


Yeah, Excel's data munging is a massive problem for a lot of CSV stuff.


Some feedback:

- I couldn't figure out how to paste a CSV from clipboard and have it parse the CSV as-is. Instead, it just creates a single column with each line as a row. I think "Split Selected Cells Horizontally" might do it, but that's locked behind the paid version, so I can't verify. Either way, this seems like basic functionality that should be free.

- Selecting seems buggy; if I click down on a cell in row 0 then drag down, it only selects rows 0-2 and then stops. If I select anything below row 0 then drag up or down, it selects everything. I'm on Linux, maybe just a Linux thing? I dunno.

- I'd appreciate a "paste from MySQL table" tool or something; I tend to copy table output from my terminal (i.e. a +----+----+ kind of table), it would be nice to not need to switch the MySQL output to CSV first if I want to paste it in here quickly to futz with it.

- It's very annoying to see tons of options and buttons that I can't use at all in the free version. Please mark those up-front in the UI/command palette as not being available, it's not fun to click something, get a popup, have to dismiss it, then hunt elsewhere for a solution because I don't have access to it. Feels a bit dickish so it makes me want to pay for it less, tbh.

- As others have said, pricing seems fair, but I'm not convinced yet so I'll wait for now.


1. If you copied from another table, it should paste as a table. If you copied a bunch of delimited text, then yes, it'll paste into a single column and you can you the Split Selected Cells Horizontally command to separate it into columns.

2. That first bug is known and for reasons I don't recall, is much trickier than one would think. It's still on my to-do list. The thing with dragging and selecting everything sounds like it might be scrolling really fast.

3. I'll consider it. Perhaps a smart paste option where it attempts to determine the nature of the text (e.g. whether it's a database table or what kind of delimiter it uses) and pastes accordingly.

4. I'll give this some thought as well. I'm not sure that disabling the commands is the way to go if the user doesn't know why they're disabled.

5. If you have any more questions, feel free to ask. Also, if you do buy and change your mind, just let me know and I'll issue a refund right away.


4. If you just disable a feature, no-one is likely to find out why it is disabled (you could put something in the tooltip, but they might not look at it). The current approach is better, if a little annoying.


I share the frustration! I have to open lots of small and large CSV files daily, with multiple encodings. Usually I'm juggling between Excel, Numbers, and Sublime text editor - depending on which one would open that file faster.

Naturally, I installed Modern CSV seconds after seeing this post.

I'm on MacBook M2, using trackpad 1) the vertical scroll is slow and un-mac-like, unnatural; and 2) horizontal scroll basically doesn't work, it scrolls a column or a half on a full swipe, but it does improve somewhat when there's less content in the cells.

I admire your work, please keep on going, there's a huge need!


You're right, Mac trackpad handling is something I need to address. It's on my to-do list.


Looks like it's scrolling in general, not just trackpad. It appears to limit the maximum scroll speed when I'm using a mouse. I take it this is because the app is programmed using some kind of non-native controls?


I noticed on Windows that if I have the window in focus and scroll, each notch of my scroll wheel scrolls 4 rows. However, if I have another window in focus and hover over the Modern CSV window and scroll it will scroll ~90 rows (not always the same for some reason). The max scroll speed is the same in both scenarios, which I tested using a free-spinning scroll wheel. After some more testing it seems a lot of programs handling scrolling differently between these two scenarios, but it is much more obvious in Modern CSV because a single notch scrolling 90 lines is quite a lot.

I would like to be able to have some kind of scroll options available, choosing between smooth-scrolling or scroll by row/column would be great. Removing the scroll speed limit would be good too.


I think you should release the Linux version as a Flatpak [1] to Flathub [2]. That way it could be install using a (graphical) software manager on many distros and it would increase the discoveribility of your program. It shouldn't be too complicated to create a package from the provided tarball.

[1]: https://flatpak.org/ [2]: https://flathub.org/


I also have a Qt based application and have customer asking about a Linux version. If I were to use flatpak.org, what percentage of Linux users is that likely to support?


I've been meaning to do this. I appreciate the reminder.


+1 for flatpak (and modern csv looks great! I will most likely buy)


Nice to see software for such usage. Also reminds me to use this Visidata which is not GUI like modernCSV but TUI https://www.visidata.org


Haven’t tried ModernCSV. Recently used TAD: https://www.tadviewer.com/ And clickhouse-local


wow, I was still using the 2018 version of TAD and didn't realize they pushed updates in 2022. The 2018 version was still better than other tools I've tried, really happy with all the recent upgrades.


When opening mobile site from India (Android,Chrome) -- the "buy" page lists prices like 19 and 29 without any currency symbol. You mean rupees, right? Take my money! ;)


Good news! It's in rubles.


I'm so glad I put all my rubles into PutinCoin.


Judging by screenshots, it seems that there are things that can be improved in UI design, for example, in "Solarized Dark Theme", the contrast in selected cell is so low (gray on purple) that the text is barely readable. To be honest, I don't like the colors from any of the themes. All the themes use grey color a lot and have low contrast.

The widgets like inputs or selects also look weird, sometimes they don't have enough padding and content touches the edges, sometimes there is too much padding. In a numeric input there are icrement/decrement buttons, but they are so tiny that I doubt they are usable even with an Apple touchpad. The text in inputs is sometimes aligned to the left and sometimes centered without obvious reasons.

The command search popup is transparent and the background seen behind it makes reading the text harder.


.... one huge thing that I'm assuming theres a fix for. How do i enable light mode? This is insane on the eyes

Edit: I found it under 'set theme'

I REALLY recommend doing light mode as the default, not dark mode. I imagine like 1% of people would actually want to look at big data as white text on black.


Really the default should be "match system". Both Mac and Windows (at the very least) have this baked into the DE at this point.


Spockula theme, like Dracula but with green Vulcan blood highlights.


Screenshot 2 shows an option to change the theme


ModernCSV is a good tool. However, from the viewpoint of a paying customer, it feels too much like a side/hobby project and that you aren't taking it all that seriously as a business. The v2 beta was announced in May 2022 with indications that the final v2 was just a few weeks/months away, as you announced the beta expiring June 25, 2022. Obviously v2 was not close to ready and paying users had to revert to v1 or to repeatedly request updated betas, each of which expired after a brief period. Your web site was not regularly updated with news or progress reports and the project appeared to be abandoned.

This is all fine for an open source project but once people are paying for a tool, using it daily, and are promised certain updates coming soon, it's not great to have that update delayed by a full year.

https://news.ycombinator.com/item?id=31576554


I agree with your general sentiment, and it's important for businesses to have good communication.

At the same time, the "Premium Personal" price is a one-time payment of $29. My basic chicken sandwich meal at Whataburger was $11 dollars.

Point being, it's obvious to me that this is 1 dude who is developing this, so I'm not going to expect flawless communication for the price of 2.5 Whataburger meals. I'd much prefer to deal with that than have something like a monthly subscription SaaS charge.


That's your opinion, and it's ok, but my employer bought the business version (4.5 whataburgers), and it bought multiple licenses (about 20 or 25 whataburgers, where's the tipping point?), plus I personally spent a lot of time (based on my compensation, likely 50+ whataburgers) vouching for a new vendor and then all the corporate b.s. like establishing purchase orders, creating new budget line items, etc, as well as justifying the purchase to IT dept which believes that Excel handles CSV files just fine.

Doesn't have to be flawless but a full year delay is a little much. I'm not even with that employer anymore.


The real dropped ball isn't the delay, but the communication (or lack there of) of same.


~$1000 USD based on the quoted Whatsburger/USD cross rate.


> At the same time, the "Premium Personal" price is a one-time payment of $29. My basic chicken sandwich meal at Whataburger was $11 dollars.

These coffee/meal prices differ a lot between countries. Similarly, there are huge differences how often people living in different countries can easily afford to eat out or buy a coffee at some café.


HN: Subscriptions are evil, one time payment is the only way software should be sold

Also HN: Why is the one developper that work on my software when he can because he can't make a living out of my one time 30$ payment doesn't ship all the features and versions I expect in a timely manner

Support contracts start at 500-1000$ a month these days, if you depend on it for your day job maybe invest in your tooling correctly...


- we spent at least 6x of your estimate

- we would have been fine continuing to use a non-expiring v2 beta, but the developer set it up differently (eventually, a non-expiring version was provided)

- we never asked for support, just to not have the software expire without a replacement available

> maybe invest in your tooling correctly...

So your suggestion is that businesses should avoid independent developers altogether? That's a great way to be supportive of the little guy.


I disagree with you there. For organisations that expect support for software over a longer period a recurring support contract is better. Otherwise you can end up supporting their never ending requests for free. Or worse they just choose something else.


You should differ between license fees (right to use the software) and support contract (your support question won't be ignored by the developer).


Almost seems like HN doesn’t have a single unified opinion on things huh!


That's a fair critique. While the app last year was mostly ready to go (aside from a few features I later decided to add), most of the issues I've had to deal with since have been related to other things (sales form, website, etc.). The license keys I gave you should work on this latest version.


Are you saying you are upset that you couldn’t use an unreleased new version, and that you had to resort to using the version you had actually purchased? Or did you purchase V2 before it was released as a sort of pre-purchase?


Our initial purchase was made while v2 was in beta and with a promise of a free upgrade to the final v2, which was expected soon. Except it was never released, until now.


Just to clarify, I did give you version 2 license keys that worked on those beta releases and this version 2 release at no extra charge, as I promised. I also changed the beta releases to not have a time limit and accept license keys instead. The beta releases I provided had nearly the same capability and quality as this one.


There was a gap between the beta expiring and me reaching out and requesting an update, which you supplied. I think the first replacement also expired and we had to make another request but I could be wrong, I'm no longer at that company and have no way to verify. Maybe not.

I am not claiming you ripped us off or that your product is no good. Pretty sure we eventually purchased 4 licenses, because it is a good product. But better communication, taking the whole thing a bit more seriously, would go a long way. You knew what day the betas were expiring, instead of leaving all the customers hanging, you could have proactively sent everyone license keys or a longer-expiring version. You could have posted a quick status update on your web site. As I mentioned in another comment, I vouched for you internally and when asked about it months later (budget reviews, etc) with still just a beta version, I didn't have any answers.


You certainly did the right thing, it sounds like you could've been more responsive though. Learning experience!


I'll give this a shot.

Heres a link to a big CSV if anyone else looking for one to test: https://github.com/datablist/sample-csv-files/raw/main/files...


Here's a complementary tool for creating (scatter) plots of the CSV-file, note that this the 100k rows example from your link:

https://csvplot.com/remote_file.html?url=https://media.githu...


Thank you, but would you mind updating your link to one that doesn't automatically trigger a ZIP file download?


Actually, nevermind, here's the link: https://github.com/datablist/sample-csv-files/blob/main/file...


Paying customer here. I LOVE Modern CSV, I've been using it for massive CSV files that have 15+ million rows of data. I (same username Chris Norstrom) was the guy that emailed a bug fix on a large CSV with UTF-8 faulty characters I was working with.

I was wondering if you had added a feature to make selecting and deleting massive amounts of rows/collumns easier.

Currently: I have a CSV with 5 million rows, I want to delete all rows that have "/gallery/" in collumn B. I have to filter that row then I have to select ALL 2 million rows and delete them. But that's so many rows that ModernCSV gets stuck so I have to delete 5,000 or 10,000 at a time if I can remember correctly. And once it starts deleting there is no message, progress bar, or sound notification when it gets done. On very large CSVs this takes forever.

Request: I wish I can somehow ask ModernCSV to "delete all rows" if "collumn B" contains the string "/gallery/" and just click "RUN" then I can walk away and go work on something else until a sound plays and I know it's done.

Maybe you have something similar already implemented?


When you select after filtering, are you doing a select all (ctrl+a or cmd+a in Mac)? Or selecting the top row and hitting shift+down to get to the bottom? Ideally, it shouldn't make a difference, but with the way it works now, it does. That's something I need to fix. If you do a select all, you can use the down arrow key to push the selection below the header.

I'll set up a similar file and see what I can do to improve this.

For now, one alternative is to filter OUT the data you want removed, then use the Remove All Filtered Out and Hidden Data command.


Homebrew install not working??

brew install --cask modern-csv ==> Downloading https://formulae.brew.sh/api/cask.jws.json ####################################################################################################### 100.0% Error: Download failed on Cask 'modern-csv' with message: Failure while executing; `/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.1.5\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 13.5\)\ curl/8.1.2 --header Accept-Language:\ en --retry 3 --fail --location --silent --head https://t6a3m9g6.rocketcdn.me/release/ModernCSV-Mac-v2.0.2.d...` exited with 6. Here's the output:

curl: (6) Could not resolve host: t6a3m9g6.rocketcdn.me


> Modern CSV is a great tool for keyboard ninjas > Suppose you want to move a column. Just select any cell in that column and hit ctrl+alt+left/right

Keyboard ninjas would prefer to have a modal editing option where instead of this abomination of a shortcut you could have much shorter sequences (as an illustration):

C,L for left

C,; for right

with an option to make L/; sticky (repeatable movements without having to press C again) or be able to hold C and just press L/; to move

(by the way, sequences cmd+A,B can't be used since they're overriden by cmd+A but it should invoke cmd+a only after some timeout when you haven't pressed B

also scrolling on a Mac is broken, and in general would be nice to limit scrolling off-screen to some # of rows/columns, it doesn't make sense to be able to scroll into an empty infinity)


Frankly, I vastly prefer modeless editing. But I don't like having to move my hand to a mouse or even touchpad (not as bad as mouse).

> also scrolling on a Mac is broken

Tell me more, pls.


having to move your hand to ctrl+alt isn't much of an improvement. Also, you can retain modeless editing if you differentiate between holds and taps, holding C is still better than holding Ctrl+Alt

Scrolling is slow by default an doesn't respect scrolling speed like most of the other apps do (e.g., fast scrolling with a held modifier using linearmouse is just as slow in MCSV2 without a modifier)


Ah, I see. Having used a space cadet keyboard for years, everything else is, ironically, easier. The SC had nice big easy to hit targets for bucky keys and other important ones like return but ironically the space they took up required more stretching. On my MBA meta/control/command are close and reachable without stetching.

Which is a long winded way of saying no one way is perfect for all. Some people even apparently will willingly use a mouse!

Thanks for the explo for your perspective on scrolling. Because i'm an emacs user I use search a lot more than actually scrolling.


they can be close and reachable, but there is no way to make them as close and reachable as a single alpha key


Looks interesting. Also worth checking out (which I use) is Visidata, similar to this but TUI.


Why does the personal license not have statistics and analysis? Those are useful features currently tied to a business license, but a personal user may not care much for having their license transferable.


In fairness the business pricing isn't really out of reach for an individual (assuming US-based tech worker).

Think it's perfectly fine to tier features and make some stuff gated at a higher price.

Especially when you consider that it's a one time purchase and also it's an editor for a standard that will never change (you won't have to upgrade in a year to support features in the latest version of CSV)


If I buy a personal license can I install it multiple times in different computers? Or do I need to transfer my license from computer to computer?


Yes, you can use it on as many machines as you want and there's no limitation on platforms either.


I've used the modern csv freeware to open 3Gb+ size CSVs and it does wonderfully well. I might just buy a license now. lol


This is good to hear, I usually use VI to open anything > 2 GB. A modern UI that doesn’t hang or crash on those files would be nice.


It really is good and you can actually have multiple large files loaded into it. Not sure how it manages that but very very good.

Just to give you some info, I just opened a csv that's 3.5 GB and has about 1.6 million rows. It took about 30-40 seconds to load nad it's only using 215 MB of ram. I have managed to crash it a few times but I think it's understandable considering the huge amount of data it's dealing with.


Can your app auto detect the CSV delimiters and also detect decimal point vs comma for numbers?

This is a huge problem when dealing with European and UK/US CSV formats (or whatever you mat want to call them) for me daily.

Excel only auto opens the format supported by the current locale, so anything else will just not be detected.


If you use Data > From Text/CSV in Excel instead of File > Open or file double-clicking it gives you more tools and does a better auto-detection.

That used to be the default behavior in Excel for years that you'd get the CSV data "wizard" but they split the behavior and "hid" the better behavior on the Data tab of the ribbon.

(The Data tab also includes From JSON which is great at avoiding building CSV files for Excel people if you can just teach them From JSON. Most don't actually want to learn and I still wish Excel would support an XLJSON or something like that file extension to give people the double-click a file they want, but with something much more useful and clearer than CSV.)


It does delimiter auto detection and for the Statistics feature, it automatically detects dot vs. comma decimals.


libreoffice calc is better with csv import IIRC


I agree libreoffice calc is better, but it does still munge some data.


Nice work. This'll make things easier for non technical people to not mess up data entry by abusing Excel. Is there a means to configure and enforce validation functions/regex for columns? Like some kind of python api perhaps? It would also be cool if data could be streamed over http(s).


Love the idea of this. Is there a feature to support views or linked records from other CSV files?

Basically, my team is curious if there's a smaller, lighter, local-alternative to Airtable, which we can collaborate on with Git. It's a big ask and maybe not on your roadmap, but something that is on our minds.


Have you tried the trick where you put CSV into SQLite?

I do that. Many, maybe linked, CSVs. Load all into SQLIte, query, filter, sort, etc.

In the repos to process we have only scripts to load and filter. The SQLite things are short lived.

Also, memory only SQLite FTW!


Duckdb does this straight from csv. You can treat them pretty much just like un-indexed tables, including queries, joins and output back to csv or parquet. its

Really nice.


That has not been on my roadmap, but I'll think about it. It sounds like it may have to be a separate product, not that that's a bad thing.


https://support.getgrist.com/self-managed/ may suit your needs.


Grist was one of the first tools we looked at but it wasn't the desktop-first app my team was hoping for. Self-hosting a service felt like a little too much upkeep, on top of all the other systems we own. I recall the data was also not git diff-able.


Desktop first made me think of Superintendant[0] which I’ve enjoyed using

0: https://superintendent.app/


There's an electron app that doesn't require hosting at all. It's a desktop solution. https://github.com/gristlabs/grist-electron


Git based database? Sounds super interesting. Are you guys actively looking for something or is it a nice to have?


Many moons and a few jobs ago, I rolled an in-house Wiki in python, that was basically a very thin web UI over mercurial. Each page was a checked-in plain text file (I think some sort of markdown flavor, but this was circa 2010, a totally different era!), each edit was a revision/commit. Things like article history just shelled out to mercurial. "Querying" was down via filesystem operations. Rolled the whole thing in a day and a half or so - less than I'd already spent trying to get a MediaWiki instance stood up!


More like a git-based log than database. We use it for requests and questions that come up on our slack. We figured out how to extract threads from slack, transform them into CSV files and then load them into Airtable.

Airtable has a great UI but the data becomes isolated from our ETL script and it means we have to re-download and export to CSV to use in other contexts.

It'd be great if we could load a CSV into a desktop editor, and then have it store certain views/linked records to other CSV files in a local "db.json" file.


If you want a proper relational database, sqlite would suit better. There are GUI tools to work with sqlite database files and sqlite has csv import and export. But that won't help much if you already have a large CSV you just need to edit a few rows on.

As for versioning sqlite data, just don't ever update or delete any rows but instead only ever insert the updates as a new row. Potentially you could add a 'deleted' boolean and 'inserted' date columns to the tables you want to version. That way you can use '... and deleted = false' to filter out old data and you also know when updates occurred.


The problem with SQLite is that you have binary blobs and you don't have useful text diffs. So you lose a lot easy history reviewing tools based on git and also easy merge/pull reviews.

The closest to a best of both worlds is to use something like JSON to store the data in git and then a tool like Datasette to build an SQLite-powered view on top of that repo.


Looks great, definitely bookmarked for the next time I'm gonna work w large csv files.

On a side note: Howcome MS failed so hard at solving this problem with Excel?

Considering it's old age and the budget behind it it would seem like Excel should be THE solution to anything to do with tabular data by now.


>Considering it's old age and the budget behind it it would seem like Excel should be THE solution to anything to do with tabular data by now.

If they change anything about how the (very broken) way they handle CSV files in Excel, it will break so many things. So they dare not change it. In fact genes have been renamed because Excel is so broken but unlikely to change:

https://www.popularmechanics.com/technology/design/a33549357...


Maybe there is no real need for supporting large CSV files? Typically large amounts of data will be stored in a database (in which case you can query with SQL), or you will be using large-data oriented file formats like parquet. Excel's CSV support is just good enough for 99% of the real world use cases.


Large CSV files do occur 'in the wild'. Whether they should or not is beside the point. Sometimes CSV is the only option to import or export data from ancient 'Enterprise' horror systems, purely because it was easy for the original developers to implement. Excel's CSV support has been demonstrated to not be fit for the purpose, as one of the other commenters here points out.

I'd not heard of parquet before today, but a cursory glance reveals it to be a stupid format. It's sold as 'smaller than csv', but size isn't the problem CSVs are solving. It's that with the CSV format it's trivial to output or read data. With Parquet it's not.

I'd imagine if you were storing data on a server it would be better to import it into a proper database rather than storing it as a file on something like S3. Even compressing a CSV file with gzip would reduce the file size similarly and in a more standardized way if that's what you really need to do.


You'd hope so, but the UK government used Excel to manage some COVID data which it then lost because there were too many rows (65k+) for the format to handle.


FYI there may be a bug with the mac version on Mojave.

I tried testing it out just now by copying a 5 million row by 14 column selection of random data, ~620MB, and it completely freezes up right after hitting command-c on my 2019 macbook pro 15. Even after waiting minutes, it's still frozen.


I would expect this to take some time, but not minutes. I'll see if I can make it quicker and if not, I'll have to add a progress bar.


I'm using this on Windows 11 and it's locking up trying to copy a single cell. As in, here's the entire repro:

1. Open Modern CSV.

2. Open any file.

3. Select any cell.

4. Hit ctrl-c.

Expected: The cell contents would be copied.

Actual: The program becomes completely unresponsive with high CPU usage and must be force-quit.

Happy to help however I can, because so far I really like this product and I want to be using it.


I've never seen or heard of this issue. By any chance, could you send me a file that it crashes on? That or a video showing the crash. You can send them here: https://www.moderncsv.com/report-a-bug/


Let me see if I can repro with a sanitized file.

In the meantime, I popped open Process Explorer. Here's a slightly abbreviated stack of the only running thread -- the rest are in waiting states. I don't know much about Qt, so this doesn't really mean anything to me. But it looks like it's maybe some sort of locking issue.

    ntoskrnl.exe!KeSynchronizeExecution+0x7db6
    ntoskrnl.exe!ExAcquireResourceExclusiveLite+0x1eb5
    ntoskrnl.exe!ExAcquireResourceExclusiveLite+0x1e4
    ntoskrnl.exe!KeWaitForSingleObject+0x256
    ntoskrnl.exe!ExWaitForRundownProtectionRelease+0x8de
    ntoskrnl.exe!KiCheckForKernelApcDelivery+0x366
    ntoskrnl.exe!KeSynchronizeExecution+0x469c
    Qt6Core.dll!QAbstractItemModel::endResetModel+0x665
    Qt6Core.dll!QAbstractItemModel::endResetModel+0x6fe
    Qt6Core.dll!QAbstractItemModel::qt_static_metacall+0x1f75
    Qt6Core.dll!QAbstractItemModel::qt_static_metacall+0x1690
    Qt6Core.dll!QAbstractItemModelPrivate::columnsRemoved+0x85d
    Qt6Core.dll!QPersistentModelIndex::QPersistentModelIndex+0x2a
    Qt6Widgets.dll!QAccessibleWidget::window+0xa91d
    Qt6Widgets.dll!QTreeViewPrivate::below+0x7be
    qwindows.dll!qt_plugin_query_metadata_v2+0x6eb3b
    UIAutomationCore.dll!UiaRaiseAutomationEvent+0x6ad2
    ...
    USER32.dll!DispatchMessageW+0x741
    USER32.dll!DispatchMessageW+0x201
    Qt6Core.dll!QEventDispatcherWin32::processEvents+0x509
    Qt6Gui.dll!QWindowsGuiEventDispatcher::processEvents+0x19
    Qt6Core.dll!QEventLoop::exec+0x19f
    Qt6Core.dll!QCoreApplication::exec+0x15d
    Modern CSV.exe+0xb841
    Modern CSV.exe+0x235267
    Modern CSV.exe+0x234812
    KERNEL32.DLL!BaseThreadInitThunk+0x1d
    ntdll.dll!RtlUserThreadStart+0x28


Thanks, though wouldn't it need a progress bar no matter what?

For those scenarios when someone tries to copy 5 million rows by 1000 columns for example.


I didn't click with ModernCSV and it lacked features. I've since found Easy CSV Editor https://vdt-labs.com/easy-csv-editor/ It has been very good


Interesting. Where does it save column size changes, since you can't add that to the absurdly simple CSV “format” (specification)? In metadata? Does it lose this settings if you change the file name behind its back? Or it doesn't try to save such things at all?


There's a setting to automatically fit the column widths based on the contents and it's turned on by default. You can turn it off and manually change the sizes, but they'll be lost once you close the program. If you change the file name externally, it'll assume the current file has been deleted. You can change the file name within the program with the Rename File command.


Surely all of those settings can be inferred from the file upon opening? Presumably the only settings you actually care about are ones that have predictable consequences for the saved document.


I work with large-ish CSV files regularly, 1M rows at times and Excel can be very slow. I tried Modern CSV and it doesn't struggle at all. Thank you!


This is a great tool. I’ve found it to be a great way to open large csv files that crash most gui programs. It’s now my favorite way to open csvs outside of the terminal.


Why does Excel autoformat data from a CSV? what is the rationale behind this? How hard it is to have an option that says "open with no formatting"?


You can import the CSV file instead of opening it, which lets you choose the data type for each column. It's clunky, but it does what you want (if you set the types correctly).

Excel is a known issue for many text data files. My least favorite Excel-ism is when it changes the names of genes to dates. Many genes recently changed names to avoid Excel-related issues. As an example: OCT4 became POU5F1, and SEPT1 became SEPTIN1. Otherwise, if you're looking at gene related data, you'd have to be very careful when looking at gene names that you didn't accidentally save the file with a few "dates" as opposed to gene names. (Which is even more confusing when you realize that dates are stored as integers in Excel).


I was doing a coursera project and it kept raising duplicates from what I thought were random strings of numbers and characters and therefore should be unique, took me a while to figure out what was happening:

https://sites.google.com/view/ryzvonusef/process

Wasted a week's worth of effort, but I learnt a valuable lesson.


Spoiler: Data read into Excel/Google sheets was interpreted as scientific notation, this caused uniqueness errors.


My least favorite Excel-ism is when it changes correct dates to american dates.

https://xkcd.com/1179/


Depending on whether you open a CSV file with Excel or import it into it, different things work and don't work. This has been a problem especially on a Windows with a locale that uses a different separator, e.g. semicolon instead of comma on a German Windows because the comma is used in numbers as decimal separator.

Unicode encoding or rather anything non-ASCII is also prone to cause trouble. The automatic conversion of quoted(!) digit strings to numbers can be worked around by appending a tab character. Line breaks within a cell are a big headache, however, and I believe the U+2028 code point can still not be used instead.


lately i've been doing things on aws cloudfront. one annoying thing, though, the unmodified format of the file cannot be opened as is by many csv viewers. for example, in excel, import from csv file won't detect the columns properly because of the first line (#version1.0).

when i open it using modern csv, the header is detected as a single column while the rest of the data is being read just fine.

using it on windows, though.


Selecting a blank column and pasting did exactly what I wanted (same value in each cell) and so - sold!


Oh God I thought you were writing, "Modern CVS" and I thought: Noooo just let it die!

Good luck on your project!


It looks cool.

What are the benefits over converting the CSV to a SQLite DB and using a DB GUI?


What UI framework are you using? It kind of reminds me of libgdx + vis-ui.


Qt 6 with C++.


Ah makes sense. It looks nice! I really have been thinking about learning QT or wxWidgets for a future commercial project.


Note to HN commenters: it's always nice to think, and express, "wow, you shipped a thing" before diving into a solid wall of criticism.

This fella has got over the biggest, greatest hurdle of them all - actually getting something out of the door - so should be commended for that. This is not to say that well thought out criticism isn't useful, of course it is, but maybe lead your comment with "Hey, amazing work on getting this far! ...if I may here's a couple of extra suggestions.." rather than "you should name it X / do Y / I didn't like Z".

Just a thought.


I appreciate that. However, I did know what I was getting into. :)


Indie developers quickly develop a thick skin. That said, it is always nice to get a compliment.


Is this downloadable via HomeBrew?


    brew install --cask modern-csv


Unfortunately this fails for me:

  curl: (6) Could not resolve host: t6a3m9g6.rocketcdn.me


Great product, bad post title!


How does it handle dates?


It can convert date and time formats. It can also write down the current date/time in whatever format you want.


Once upon a time, I went looking for a MacOS CSV viewer and was surprised that there weren't one or two widely used, clearly awesome options. I went hunting.

I downloaded Modern CSV, hopeful that it would be a quick and easy solution. I really wanted to like it! I even paid for it. I made it my default app for opening CSV files. But after a week or so, I gave up in frustration. It has lots and lots of editing functions (like, 50+ editing commands!) but first and foremost it has to work as a viewer, and it doesn't. It has too many design decisions that don't make sense for a viewer.

Take, for example, the most common use case: I want to see what's in a CSV file, and it has a header row, because the vast majority of CSV files have a header row. What are the things I want to know?

    1. What columns are present?
    2. What do the columns mean?
    3. What does the data look like?
Let's see how this goes.

I. Headers

The first thing that stands out is that, by default, Modern CSV highlights the headers, but — misleadingly — doesn't freeze them. What's the point of headers if they are not frozen? As soon as I scroll, the answer to #1 is gone. Within the first 10 seconds of using Modern CSV, I find myself hunting for the "Freeze Headers" command.

I right-click on the headers, but it's not among the 25+ items in the context menu. I have to scan through all the pull-down menus, to find the single most commonly needed command: it's buried on the second level of menus, after 27 other menu options. 27! It is also mixed in with a few other similar commands, so I have to think about which one to pick:

    Toggle Highlight Header Rows
    Toggle Highlight Header Columns
    Toggle Freeze Header Rows
    Toggle Freeze Header Columns
    Toggle Freeze Header Rows and Columns
What are "Header Columns", by the way? In my whole life, I have worked with many, many CSV files and I've never seen header columns. Clicking Toggle Freeze Header Columns doesn't seem to have any visible effect.

Ah, now I've found "Toggle Freeze Header Rows." I will need to do this 99% of the time I open a file. Does it have a hotkey? No. I will have to navigate through two levels of menus to find item #27, every time.

II. Hotkeys

I know I won't last long if I have to navigate two levels of menus every time I open a file, so I look for a way to add a hotkey. Unlike every other Mac app, Modern CSV doesn't have a preference menu in the usual place, and Command-Comma doesn't open the preferences.

To add a hotkey, you must edit a CSV file of hotkey definitions, which means learning how the file is organized and the syntax for specifying hotkeys. Deviously, the column with the hotkeys in it is not the one you edit. Editing this column does nothing; you must instead edit the empty column next to it. This is explained in the instructions, but of course you can't see them, because they disappear when you scroll down to find the row you want.

Also, as you scroll down to find the "Toggle Freeze Header Rows" command, the column with the command names occasionally disappears off the left side of the screen because Modern CSV insists on letting you scroll over to column 49 even though there are only 3 columns in the file. Which brings me to...

III. Scrolling

Having solved this, my the next action is to scroll around to look at the data. And here I am stymied again: the scrolling response is maddeningly hard to control, for four reasons.

First, vertical scrolling happens at a constant speed. It doesn't matter how slow or fast you swipe your fingers on the touchpad; there are exactly two speeds: zero, and about 60 rows per second. If you are scrolling down to row 1,000, you must swipe and swipe and swipe continuously, and wait for ages until you get there. The constant speed also means that if you want to adjust the view up or down by one or two rows, it's impossible; the view will jump by 7 or 8 rows no matter how tiny your swiping movement.

Second, scrolling will continue coasting after you stop, for some distance. This distance varies, but it is too far; when I swipe only a small amount, the scrolling continues for about 30 rows. I tried a few tiny swipes, gradually swiping a little more and more, to try to figure out the algorithm; and it appears that the coasting distance stays at zero for the smallest swipes, and then suddenly jumps from zero to about 30 rows. Again, the discontinuity makes it very difficult to control.

Third, horizontal scrolling only moves by whole columns. You get no visual feedback until the view jumps by an entire column, and columns in CSV files can often be quite wide. This makes it very hard to control how much you scroll, and again, hard or impossible to shift your view by just one or two columns.

Fourth, Modern CSV always shows 50 columns, even if your file has fewer. That means there will often be a giant empty space off to the right of your data. And because there is no feedback until the view jumps by an entire column, it's easy to accidentally scroll over and get lost in that empty space.

The first of these problems alone is enough to make me want to quit the app.

IV. Filtering

If the header doesn't make it obvious what's in a column, a good way to understand it is to see what values are present. The Filter feature in Google Sheets and Excel is great for this: you get a dropdown menu in each column that shows you a list of unique values in that column, which you can tick on or off to apply a filter.

Modern CSV doesn't have this feature. "Filter Selected Column" is in the context menu for a column, which is great, but it doesn't work; it's a paid feature. This is basic stuff; you can't explore a 10,000-line CSV file without filters.

In a licensed copy of Modern CSV, "Filter Selected Column" brings up a dialog with ten lines of syntax instructions for me to read, and a blank text field to enter a filter expression. I don't want to have to read syntax instructions to apply a filter. If the window is a normal width, I can't even see all the syntax instructions, because they are cut off and cannot be scrolled into view. Resizing my window won't help; I have to close the dialog, resize my window, and open the dialog again to read the instructions.

The filtering syntax has three different ways to specify a range of numbers (>=20 & <=30, or 20+ & 30-, or 20–30), but if I want to filter for a particular string value that appears in half the rows, I have to retype the string into the box. In Google Sheets or Excel, you just tick a checkbox.

—

The whole app feels like this. Sophisticated operations are possible, but the most basic, common operations don't work properly or are unnecessarily complicated. I was really sad to go back to Excel because it's so heavy and slow, but I went back to Excel.

I know this is a long rant, and no one asked for my advice. But I hope this is helpful information. If I were to advise the author, I would recommend:

    Make easy things easy.
    Make scrolling work.
    Choose good defaults.
    Provide filtering in the free version, and make it easy.
That would bring me back to Modern CSV.

—

All these things become obvious if you watch someone try to use the app for the first time. Perhaps even better advice would be:

    Stop adding features.

    Watch someone use the app for the first time, for
    a real-life use case with a real-life data file.

    Just watch.  Do not say anything.  Record every 
    frustration they encounter in the first ten minutes.

    Then, with maniacal focus, eliminate every one of
    those frustrations and test again and again until
    the first-use experience is completely smooth.
I guarantee this will get you more paid users.


Many of your complaints were addressed in version 2. These include:

- A preferences window for settings, keyboard shortcuts, and file extension options

- A Unique Values feature

- A File Metadata pane to easily set up things like header rows

- The command launcher width changes as the window width is changed.

A few were addressed in version 1:

- Persistent storage of settings set via command

- Headers rows frozen by default

Header columns work the same way as header rows, just vertically. I use consistent verbiage for clarity. If they appeared to do nothing for you, the reason is probably that the number of header columns is 0 by default. I haven't seen any confusion about it since adding the File Metadata pane.

There's a Minimum Number of Table Rows/Columns setting that's set to 50 by default. You can set it as low as 1. There's also a Number of Buffer Rows/Columns that will add rows and columns beyond the data. It's set to 2 by default.

I'll take a good look at scrolling on Mac. I haven't seen problems with my setup, so I'll work to reproduce what others are seeing.


[deleted]


While we're inventing a CSV 2 standard, can we please get rid of the weird quirks like allowing newline literals inside quoted values? It makes parsing about 5 billion times harder because you cannot use the readline() functions available in various languages to parse CSV.

Newlines in values should be explicitly escaped, e.g.

    foo,2.0,bar,this\nis\na\ntest,blah




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: