Take this strawman for instance, how could you find the bug in 
  the following code without the accompanying comment?

  // Print every other line of the array to the console
  for (var i = 0; i < myStringArray.length; i++) {
      console.log(myStringArray[i]);
  }

  This shows how important intent is to a program. Yes, the  
  example is trivial, but without that comment, how would we know
  that it's wrong?

In the real world, the answer is almost always "the code that's actually been executing, rather than the comment that hasn't," which is why it's a good idea to minimize commentry of that type, to prevent confusion on the part of the reader.

So if you're doubling-down on literate programming, how do you address that problem? That article says, "Inconsistencies between the code and the prose are themselves errors in the program and they should be treated as seriously as a buffer overflow," but does Eve actually do that? It's hard for me to see how it could, in the examples given.

(If the idea is simply that this is extra work for the programmer to do -- that they need to describe every implementation twice, once in code and once in English -- I submit that it's not very human-focused at all, if those humans are programmers.)

Comments should explain things that are not obvious from the code. Comments should be things like

// Note: PCI-DSS requirements apply below

// Must check status register and FIFO to determine completion due to flaky hardware

// Algorithm below is modified Knuth-Morris-Pratt

// This is O(scary), but seems quick enough in practice.

(Now, if someone lets me put images in comments, that would be amazing. Good for state transition diagrams and random scribbles)

But there's ways to mix the two: Python doctests is one. The lp approach is to "escape" the code, not the comments. I really do think some richer data structure than flat text files is needed - simple multi-/hyper-text seems like a minimum. Not only links (most ides have this: hoover to display help, click to goto definition etc).

Why so few seem to successfully add vector graphics and diagrams I don't know. I guess jupyter / ipython might be a rare breath of air. But who really wants to maintain 10k locs of code and 15k locs of tests in ipython notebooks?

We need something more (lively kernel might be more interesting here - webdav for persistence Web browser for run/edit/view - I'm not sure if the required power Canberra realised any simpler).

The best example I'm aware of is probably Smalltalk coupled with monticello version control and a solid object database like gemstone/s (I'm not aware of a real open/free workalike for the last part).

That's not not say relational or logic databases aren't useful - but it's more difficult to manage the data, query code and program code in one integrated system if the database takes the form of an external server. That said, Ms visual studio and the SQL db gui does a pretty good job of presenting a somewhat coherent environment.

I used to be strongly in the Linux/Unix camp - thinking that the datastore we call the filsystem is a good abstraction. But I've come to believe that even the strong legacy of user familiarity isn't a good enough reason to stick with it. Even if we were a little bit more serious and at least went all plan9 - rather than the half-hearted state of Linux/Unix (everything is a file, or a database file, or a binary file or an archive of a filsystem or...).

That said a "filsystem" might very well still be a decent building block for a higher level system (with decent search, for example).

But yeah, I don't think jupyter is the be all, end all of development. But it is an interesting way to move legacy programming environments forward in a low friction manner.

The comments should be the things that you would tell a new team member during a pair programming session. You assume she knows what the syntax of the language is, but you don't assume she knows the business intent behind the code or the history of trial and error that led to its current state.

That's exactly what JetBrains MPS (https://www.jetbrains.com/mps/) does. Even better: the state transition diagram is the code. Some of my colleagues are playing with it - not sure if it's used in production yet.

In his case, his algo said X, and his code did Y.. very easy to see the mistake.

In your case, lets add a comment

// Note: PCI-DSS requirements apply below

Now 3 years later, the law changes and the requirements do not apply. So you are at the same situation. Code does X, comments say Y. Which is right?

Not a very easy to keep comments and code in sync

But at any rate, why is relevant, and code doesn't express the why.

Comments can be useful for annotating algorithms or referencing stack overflow though.

Requirements are usually exogenous to the code.

1. Specs&design docs and code should be in separate files, because I believe the separation of concerns should be applied there. That's indeed the opposite of literate programming.

2. There should be two-way links between documentation and code: in the code, one should have links to the spec; and from the spec, one should have links to the code.

3. If the specs or the code changes, those links should be displayed in a different way to warn the reader that things are potentially not in sync. How to do that: check if the links point to the latest version. The maintainers have to update the links to remove the warnings.

Specification and design/implementation are not separate concerns. They are dual.

A sufficiently detailed specification is an implementation. Prolog does this (and Eve has a very similar feel).

As engineers, we traditionally work declaratively at the top of the "V" and imperatively at the bottom of the "V" -- but the reasons for this are largely historical/cultural.

We could (in theory) carry out the analysis/refinement process using entirely declarative notation.

The problem domain has primacy. Analysis separates problem-domain concerns and the duality takes care of the translation between problem and solution domains.

(OK -- so this is basically just a reiteration of the thesis of good old-fashioned AI -- that with a sufficiently powerful theorem prover and a sufficiently large and detailed knowledge base -- solutions will just pop out of a largely mechanical analysis process -- and I'm pretty sure this isn't at all trendy right now ... so I should relegate this to the "thinking out loud" bucket ...)

Sometimes a piece of code needs to be highly optimized to the point of being basically unreadable, at that point it's probably worth adding explanatory comments that you would normally avoid.

As another example, what if it were a method name instead of a comment?

  function printEveryOtherLine(myStringArray) {
    for (var i = 0; i < myStringArray.length; i++) {
        console.log(myStringArray[i]);
    }
  }

I don't know about you but I certainly know that if a printLine function accidentally does something else, it's definitely the code that's wrong and not the function name.

"Proven in use" and backwards compatibility is a bitch.

  def sumAllNumbers(nums: List[Int]): Int = {
    nums.filter(_ % 2 == 0).sum
  }

So the name definitely helps, but it's not conclusive.

Also addresses are not like names. As with names in general, a name can refer to more than one object. (Consider modules, for example.)

I don't agree with this. I think often a function will drift from its name if functionality is added to an aspect of its implementation or because of refactoring.

that's a rosy way to say, names entail a message.

An Address can also be computed, your argument is invalid.

This does not happen in your function name example: if the program requirement changed to print every line, no sane programmer would change the code in this function, they'd write a new function printEveryLine() instead and use that. So you can be pretty sure this is a bug.

I'm not sure I understand the distinction you are making here, so I'd like to try and understand. From my perspective, function names are just another form of comment. After all, in many languages, as soon as you press "compile", your function names are mangled into something a machine can understand.

So you feel that programmers are more inclined to treat the function name as an authority, as opposed to a comment. Is that an accurate? I'm curious what lead you to this conclusion.

I think this is a reasonable claim, even though as you've said there's nothing strictly enforcing this behavior. Function names are generally highlighted by the editor/IDE, and the developer is forced to stare at them and at least begin to type them out while calling them. They're brief and repeated over and over again in the source code. They're just harder to ignore. Comments on the other hand only appear once per comment, tend to be italicized and greyed out by IDEs, are generally at least one sentence in length and can span multiple paragraphs, and are never going to be directly referenced from within the code itself. It's still very possible to change a function without updating its name to suit its new purpose. But I know for me personally I'm much more prone to missing comments that I need to now delete or reword. I have to acknowledge the function name when I write code calling it or read code referencing it, but comments are just sort of there, hovering in the background. It's easier to read them once and then tune them out and forget that they're there.

Indeed, someone else will probably end up duplicating the effort of writing the same function over again—but with an accurate name this time—because they aren't aware that the functionality they want exists in the codebase already.

And this is all an implicit consideration made by every programmer, every time they define—or later modify—a function. We all know that we'll "lose track of" functions if we don't call them something memorable for their purpose. So we put thought into naming functions, and put effort into renaming functions when they change. (Or, with library code, to copy-pasting functions to create new names for the new variant of the functionality; and then factoring out the commonalities into even more functions. We go to that effort because the alternative—a misnamed function—is almost effectively beyond contemplation.)

Unfortunately many programmers seem not to be sane.

If this were only true.

His complaint is that it isn't being done here, so all the prose is jut going to get out of date and incorrect, and meanwhile you have to write everything twice for no benefit.

I'd love to be proven wrong, but I think the problem is that this goal itself is mistaken. If we can formally specify something using a language X, this necessarily becomes a language not "for humans" -- i.e. a formal language that doesn't follow the rules of natural language and that we must be trained in. Natural language is notoriously bad at unambiguous specification.

I could see potential in a function-level spec in the form of an append-only transcript, where only one line can be added at a time. So you (with your requirements-specifier hat on) write a line of spec; and then you (as the programmer) write a function that only does what the spec says, in the stupidest way possible; and then you put on the other hat and add another line to the spec to constrain yourself from doing things so stupidly; and on and on. The same sort of granularity as happens when doing TDD by creating unit tests that fail; but these "tests" don't need to execute, human eyes just need to check the function against them.

---

On another tangent: I've never seen anyone just "bite the bullet" on this, acknowledge that what they want is redundant effort, and explicitly design a programming language that forces you to program everything twice in two separate ways. It seems to me that it'd actually be a useful thing to have around, especially if it enforced two very different programming paradigms for the two formalisms—one functional and one procedural, say; or one dataflow-oriented ala APL or J, and the other actor-modelled ala Erlang.

Seems like something NASA would want to use for high-assurance systems, come to think of it. They get part-way there by making several teams write several isolated implementations of their high-assurance code—but it will likely end up having the same flaws in the same places, given that there's nothing forcing them to use separate abstractions.

I suppose that English is imperfect and could use improvements. Ideally those two would converge. That would solve problems about the interpretation of laws, as well. How would that convergence work? To answer that you'd have to know how language is acquired in the first place. I suppose some form of self-reference in the language that mirrors some of Chomskie's stipulated universal grammar, if it exists.

Imagine if you came across this in actual code. You might first check if the code was changed recently and by whom, and whether the comment or the code came first. But that still wouldn't tell the whole story. To get it all, you would find out who checked in the code, and what their original intent was.

What we're advocating is to treat source code more like a design document. It's a collection of thousands of design decisions all informing the specification of the final product, the program.

> If the idea is simply that this is extra work for the programmer to do -- that they need to describe every implementation twice, once in code and once in English

Right, so the point is that prose is not redundant. As Knuth put it literate programming is "a mixture of formal and informal methods that reinforce each other." When we communicate to one another face-to-face, we use gestures, expressions, intonation, etc. to articulate an idea. On the internet, when we communicate with just text, much of my meaning is lost forever and never apparent to anyone who reads this.

Programming is much the same way. The code only tells you so much about the program. Without any context, readers are often disoriented and confused by new codebases. If no one is willing to read your code, no one will ever contribute to it. GitHub and SourceForge are veritable graveyards of amazing yet abandoned projects that will never see another contribution, because their codebases are so inscrutable. I know I'm guilty of this as well; when I revisit old codebases, I hardly know even what I was thinking.

Anyway, I think there's a lot to explore here. I think what you raise is an issue, and it will always be important to address, but I really believe that literate programming is important and can lead to better code. So we're going to see where we can take it with Eve, and I hope you'll give it a shot!

Why? Where is the prose? What makes programs supposed to be written in Eve different from Eve itself?

Our Eve source on the other hand is wonderfully literate and it's been a really great experience. Here's the code that makes the inspector work [3].

[1]:http://programming.witheve.com/deepdives/literate.html

[2]: https://github.com/witheve/Eve/blob/master/src/runtime/join....

[3]: http://play.witheve.com/#/examples/editor.eve

Python is just "ugly" C underneath, containing for example a 1500 line switch statement, but this does not in any way change the fact that it's a great piece of software.

I'd also like to say that if code is easier to read in the first place, there will be more eyes on it, and discrepancies will be resolved sooner. In most programming environments, a comment might describe a part of a much larger function. But the output of this code is far removed from the actual source. So discrepancies between comment and source are harder to spot.

It Eve, we intend these documents to be for more than just programmers. Who again, might not understand your code, but would understand the written prose and see the output /right there in the code/ that contradicts it. A bug like this would be brought to your attention much sooner than if your code was simply hosted as on GitHub.

Actual thing that will happen, if you're not lucky: Same as above, but they don't delete the obsolete paragraphs.

But I'm not sure why it would be considered beneficial to withhold knowledge of intent in favor of purely what's happening; except maybe to junior/support staff that aren't likely to understand the motivation behind a developer's intent. Perhaps, because of the concern that code gets maintained and related commentary doesn't?

The comments will tell you what was intended at some point regardless of what is happening. In any mature code base that includes these "what not why" comments, the comments will inevitably document old intents that have since changed.

> Maybe I'm biased because I'm pretty proficient in the business domains I specialize in, so I can understand, conceptually, what the application is suppose to support and consider the validity of the programmer's intent as well as the code's function.

Given that, why do you need these kinds of comments at all? You already know what the code is supposed to do anyway, why introduce a comment that you might later forget to update, leaving less knowledgeable colleagues confused?

Because in large, logically complex systems, especially large complex systems that are sold to many customers like my area of expertise, ERP systems implementation & development, there are many, many right answers (there are many wrong answers, too). Take inventory costing; my current project is adding an industry specific inventory handling methodology to an existing ERP system. Consider the inventory unit costing: there are multiple methods (standard, weighted average, FIFO, retail method, etc.), there can be different derivations and components (purchase cost, landed costs, material costs, direct overhead cost, indirect overhead costs) and to top things off, not only do systems implement more than one of these at a time, sometimes different of these can be useful concurrently depending on context: if I'm analyzing my vendor costs per item, I might not want my overhead costs in the picture, but I still need overheads in the capitalized cost of the item when determining Cost of Goods Sold when I ship product out, for example. In software, if I see a function like (not real, way oversimplified pseudo-code):

function calc_item_trans_cost() { total_item_trans_cost = item_cost + item_freight_cost return total_item_trans_cost }

I might not know exactly where or how to use this properly, even having some expertise. Was the developer trying to get at a landed cost? Are they expecting that this would only be used in the invoice matching process and shouldn't really be used outside of that? I might get that contextually from surrounding code or from the (various) contexts that the function is called in, but now I have a research project trying to figure out why they created this so that I don't use it in inappropriate contexts... or I can re-invent the wheel (not a good strategy). My professional knowledge will tell me where to look for other telltale signs to figure out the answer, but not immediately just from looking at the code since there are multiple valid answers. If I had:

// Function for determining item costs for three way matching. // We record freight costs separately from item material costs, // but invoices have no such breakout, so we do the math here // to facilitate the matching process.

function calc_item_purch_cost() { total_item_purch_cost = item_cost + item_freight_cost return total_item_purch_cost }

now I get it. It's not that the system has a simplified view of landed costing, but there is a specific purpose that was trying to be accomplished by this function.

> You already know what the code is supposed to do anyway, why introduce a comment that you might later forget to update, leaving less knowledgeable colleagues confused?

Most of the time the comments in the code I work with really don't change much in regard to intent. If the comments get too close to describing the logic, then yes, the comments don't age well as the logic that achieves that intent is improved, but those are the comments that are probably not needed. The contextual comments, however, are important (even for me to understand what I wrote after some time has passed).

Even so, I was speculating/rationalizing on why people might hold that somehow "ignorance is bliss" when I spoke of junior staff. I don't hold that belief myself. I haven't found that withholding information even from junior/support staff is better than giving more complete pictures. I find the informed person much more useful that the person guessing at what it could all possibly mean and fit together.

Unless you write the comment, then the code, then change the code and don't update the comment. Now if the code is wrong, both the comment and code are wrong.

In current languages letting code handle the 'what', and comments handle the 'why' works well enough. Not saying it is a perfect system, but the idea of the comment being expanded to include the 'what' as well doesn't seem like progress in the right direction to me.

If you're working with a relatively simple application where the information the application deals with is conceptually simple: you're absolutely right, explaining what and then having code that has a finite set of possible intents would just be wordy. On the other hand, I gave an example of ERP system inventory costing algorithms in another answer in this general thread where the complexity is different: cost can mean different things under different circumstances and different processes. We try to generalize these algorithms into lower lever functions, but you have to understand the what to know how it's appropriate to use the function... or if it is actually achieving it's goal: it may be coded in a way to be valid in all but the use cases targeted. There simply is no one size fits all answer to how to appropriately comment code (or if to do it at all).

As for bad comments and comment rot... any code, commented or not, can be written poorly or not maintained properly. Yes it's an extra moving part, and if you're not going to be as diligent about documentation as the code then, yes, often times your better without it. I think, however, it's better to force the issue. I don't think you need freestanding documents but by god if you have comments and I see code in a code review where the comments have been neglected: reject that submission.

The odd or even lines? I don't think English is going to replace programming languages anytime soon.

A bug: that flappy bird program listing caused the page to flicker for me. It seems like the content is being repainted over and over. I'm on a Nexus 5x using the stock Chrome. Scrolling that block a ways off the screen made the flickering stop, but it came back when I scrolled back up. Hope this is helpful!

Every time I delve into Xerox PARC and ETHZ papers, along side my own experience, I envision the days when our programming environments would be all like this, even for systems programming like tasks.

All the best for the road ahead.

Maybe this is just an unsupported use case but I think it's because the browser window isn't wide enough so the text on the left gets cut off. I have my browser window filling up half of the screen.

EDIT: works now!

Firefox 49.0.2

Makes it very hard to read.

Awesome stuff though, excited to see where it goes.

I opened the quickstart in Chrome and went all the way through without a hitch until the very end - the last code block doesn't appear to do anything. Clicking submit doesn't update the visible output, nor does it clear the form as the code shows it should.

I'm thoroughly liking the environment other than the hiccups though. I'm not familiar with datalog, but have dabbled a bit in prolog and always liked the way unification felt.

EDIT: Also: base.js:7035 Blocked a frame with origin "https://www.youtube.com" from accessing a frame with origin "http://programming.witheve.com". The frame requesting access has a protocol of "https", the frame being accessed has a protocol of "http". Protocols must match.

EDIT2: Why does opening the chrome console fix the text cut off issue.... this is so weird.

Fix confirmed!

The set operator sets a value on a record: http://docs.witheve.com/handbook/set/

These values can be literals, or even other records.