Hacker News new | past | comments | ask | show | jobs | submit login

I'm interested, can you explain more about the merging idea?

To clarify the requirement, history could be a JSON array of objects:

    [
        {"cmd": "git checkout", "when": 1234 },
        {"cmd": "vagrant up", "when": 4567 }
    ]

To append an entry to this file and keep it valid, one must locate the closing square bracket and overwrite it. That work is what I hope to avoid.



Why not use a “stream” of objects?

  {"cmd": "git checkout", "when": 1234}
  {"cmd": "vagrant up", "when": 4567}
Not sure about other languages and libraries, but Go supports this out of the box[1]. And while we're at it, why not CSV? That can be processed with awk.

  #cmd,when
  "git checkout","1234"
  "vagrant up","4567"
[1]: https://play.golang.org/p/sTN9z4Kv3DB


CSV is fine for simple cases but has issues with versioning (adding new/optional fields) and nested data like arrays.

The object stream idea is apparently supported widely and seems pretty strong. Thanks for the suggestion!


I use JSON streams a lot with command line tools. Keep in mind that it’s limited in that each object must consume a single line. This allows you to recover from syntax errors in a single entry; each line is a fresh start.

Have you considered SQLite? I know it’s not a friendly text format, but it alleviates a lot of the issues with the “append to a text file” approach, such as concurrency. It’s great for this sort of thing.


Loosely running with your example:

A file already exists with this content:

[{"cmd": "git checkout", "when": 1234 }]

Another tool wants to add a setting/element/etc, and simply creates a new config object with only the change in question included and APPENDS the existing config.

[{"cmd": "git checkout", "when": 1234 }][{"cmd": "vagrant up", "when": 4567},{"comment": "Comments, notes, etc are kept even if they don't validate to the recognized configuration options."}]

A configuration validator / etc loads the config and merges these in state-machine order, over-writing existing values with the latest ones from the end of the stream of objects and then determining if the result is a valid configuration. (Maybe file references fail to resolve / don't open / there's some combination of settings that's not supported...)

    [
      {"cmd": "git checkout", "when": 1234 },
      {"cmd": "vagrant up", "when": 4567},
      {"comment": "... actually kept as above"}
    ]




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: