I am a math professor with a scanned exam grading workflow that I hacked together as Bash scripts using various open source command line tools. I feed all the exams through a sheet-fed scanner, decode bar codes to identify problems and students, add radio buttons for entering and tracking scores (0-6 per problem), and create PDF "books" per problem for grading and annotating.
Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Instead, after each of several provisional passes I recreate the PDF "book" for that problem, with a chapter for each score, and students randomized within each chapter. In the same spirit as "checking your work lets you work three times faster" this is actually both more consistent and faster that a single pass over paper. Almost all of my attention is on the math, which I'm good at, rather than locating problems and finding again the ones I know I misgraded, which I'm not good at.
Then each student's exam needs to be extracted from these problem PDFs, scores recorded, and annotations frozen.
There are cloud services for grading. They're hopelessly primitive, with cloud lag. Like a gamer, I used to reject wireless mice because of the lag. I reject these services. I can grade everything myself faster than using a team of grad students, with the right local tools.
The PDF format is a morass. My hats off to anyone who will work with it. There are many evolutionary layers and no formal specification or verification; one tests a PDF by seeing if most programs accept it.
It's time for me to rewrite my grading system in a modern scripting language, so others could use it. I prefer Ruby, but that's mainly to stave off boredom when I'm not using Haskell. I can use Python. This would permit a more robust workflow, such as adding late exams in mid-grading without losing grading in progress.
I can't find documentation for Borb, to check off the list of features I'd need. I suspect from this being a one-person project that I might need to continue to patch together external tools.
You should consider looking into Gradescope (Gradescope.com). As a former TA, I can attest to it making grading much more pleasant and streamlined than it would be otherwise.
PDF does not use Postscript, and is not Turing complete. Its drawing model is based on Postscript's (with additions), but its instruction set is focused on drawing, and can't do programming. Here is the instruction set, with equivalent Postscript commands where applicable: https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PD.... Most parts of the Postscript language have no PDF equivalent.
Other things like JavaScript and Flash can be embedded, but they are extras on top of the document.
It's based on postscript, like json is based on javascript. I don't believe there are any control flow (or even general arithmetic?) instructions in the content streams, so I don't see how it could be Turing complete. It's just a sequence of drawing and transformation commands, like SVG.
I presume the extensions to embed javascript are Turing complete, and IMO do not belong in a PDF file.
I've also heard that some of the embedded font formats have features that are Turing complete, but I don't know the details on that.
If there is a way to implement a Turing machine in PDF, outside of the fonts and javascript, I'd love to see the details. (I know somebody managed it with just the macro expansion bit of TeX.)
See dunham's answer. Anyone who has seen a PDF file in text form would swear they're looking at Postscript. There's a common intersection that's identical.
Very roughly speaking (this is a semantic debate where everyone is wrong from someone else's perspective), a PDF file is a restricted subset of Postscript, with added indexes so one can render pages in the middle without having to process the code from the beginning.
The hardship in generating PDFs from scratch is getting those indexes right. It's far easier to convert a Postscript file using standard tools.
Sorry, I should have elaborated in my initial comment.
I was briefly involved as a developer several years ago (as part of my bachelor's thesis). At that time, it was mostly beta-quality, but it was already in use by multiple professors for grading. I haven't been involved with the project since, so I'm not sure about the current status.
I think the homepage [1], which you linked to and where it mentions that it's still a prototype, is at least somewhat outdated; it has a screenshot of a very old version of the software. At least the 'support' section still looks accurate, though.
If you're interested in using it, I would advise getting in touch via the Mattermost channel or mailing list (both linked to from the homepage [1]) and asking about the current state of the project. Tell them Jamy sent you :)
If you want to do something like this in Ruby, have a look at HexaPDF - https://hexapdf.gettalong.org/ - which provides a full-blown implementation for reading and writing PDFs and is quite mature already (n.b. I'm the author).
It is licensed AGPL+Commercial but if you just use it for yourself, this does not matter as you can use the AGPL.
Haven't tested this lib, however be careful before including it in your project because of its license (it is dual licensed agpl/commercial). This means that you can use it only if your project is GPL or else you need a commercial license.
On the other hand, the reportlab pdf generation library (which is what I actually use) offers a permissive language in its open source version (and a commercial reportlab plus version), so it can be included in all kinds of projects.
Page numbers are part of the PageInfo object within a PDF, but it looks like currently this is generated naively [0], so I don't think multiple numbering schemes are currently supported by this library.
It is literally impossible for all PDFs, since some of them may not have any kind of semantic structure and consist of a set of graph bitmaps laid out at specific coordinates to make up blocks of text.
The best PDF bug is when the linker between the Adobe character value and the font/language is broken and you get random Unicode like values with no way to connect the two.
It's not very effective against anyone determined though. You can OCR easily. You can also rebuild the character mapping from the shapes of the glyphs, and in most languages there are few enough that you can even do it by hand.
You can OCR if you're using a Latin script with few if any accent marks. Depending on your OCR engine, I suppose you could do Cyrillic too. Other scripts, not so much. (And yes, I'm a computational linguist, so we deal with non-Roman scripts all the time, particularly Arabic script. But I suppose that's not a problem for most people here :-).)
There might be some Latin script fonts that cause problems, but I haven't looked into that very much--I do recall we had problems with an italic font.
When I came across this I already had a pristine copy of the font, so I just compared the program for each character to determine the mapping. (I was automating the decoding.) I agree that there is little to no security there.
But the point that I was not so clearly trying to make was that sometimes the messed up encoding is intentional and not a bug.
There are a few PDF Python libraries and open source programs out there, but as far as I can tell all lack one feature: signing. If anyone knows of a open source tool kit or library that can sign, I'd be most appreciative.
This is awesome! I've been looking for something that could be a link between PDFs and Instapaper[0] for a while. This looks like it'll be perfect to build such a tool with.
I have a PDF of a hymn book that I want to convert to 2-up so I can use it that way on my reMarkable which only supports single-page display and not two-page spreads; but I also want all the internal hyperlinks (to and from a table of contents) to keep working. I haven’t found any software that seems capable of doing this (though I’ve only looked at FOSS; wouldn’t surprise me if something Adobe could do it). The closest I seem to have found is qpdf which might be able to do it with some programming effort.
Is that sort of thing going to be in scope for this library’s editing capabilities? (“Editing PDFs” is such a broad, open-ended thing.)
So how does this compare to the python bindings of mupdf? Which IMO is the most featureful module to manipulate PDFs in python (I'm a bit buffled by all the comments that something like this didn't exist before).
So, are there decent lower-level (e.g. C++ or even C) libraries for doing this, which this library wraps? Or does this actually do the nitty-gritty PDF innards itself?
As for myself, I've not had to automate work on PDFs, luckily; for manual manipulation and annotation I've found Xournal++ sort of useful (https://xournalpp.github.io/). Inkscape can also be used with some questionable PDFs.
Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Instead, after each of several provisional passes I recreate the PDF "book" for that problem, with a chapter for each score, and students randomized within each chapter. In the same spirit as "checking your work lets you work three times faster" this is actually both more consistent and faster that a single pass over paper. Almost all of my attention is on the math, which I'm good at, rather than locating problems and finding again the ones I know I misgraded, which I'm not good at.
Then each student's exam needs to be extracted from these problem PDFs, scores recorded, and annotations frozen.
There are cloud services for grading. They're hopelessly primitive, with cloud lag. Like a gamer, I used to reject wireless mice because of the lag. I reject these services. I can grade everything myself faster than using a team of grad students, with the right local tools.
The PDF format is a morass. My hats off to anyone who will work with it. There are many evolutionary layers and no formal specification or verification; one tests a PDF by seeing if most programs accept it.
It's time for me to rewrite my grading system in a modern scripting language, so others could use it. I prefer Ruby, but that's mainly to stave off boredom when I'm not using Haskell. I can use Python. This would permit a more robust workflow, such as adding late exams in mid-grading without losing grading in progress.
I can't find documentation for Borb, to check off the list of features I'd need. I suspect from this being a one-person project that I might need to continue to patch together external tools.