Because this is coming from nim and nim design goals are to be efficient and elegant I can not help but notice the excessive bloat. The main forum page is 4MB of data (~700kb compressed) to basically communicate 3KB of information (plus avatar images which should not be more than few kb total anyway).
This is still better than original Discourse which for this same thing needs 6MB of data (1.2MB compressed). However Discourse does not claim to be efficient or elegant.
This kind of web performance engineering attention to detail will go a long way. The benefits for users on slower connections are particularly important. And as an engineer, showing few KBs of information with few MBs of data should bother you. Your information to data ratio is less than 1%. Excessive reliance on JS does not help. The best way to think about JS is as a feature enhancer not a feature requirement.
Having said that I think it is absolutely great that you are doing this and I hope we can get efficient and elegant forum software. This will certainly help nim adoption if executed in the way that is true to your values.
Edit: A competitor has been mentioned in comments:
They are showing similar amount of information in "only" 100KB of total data. And 80% of it is CSS (which does seem excessive for what it does, I am sure 5KB of CSS would do the same job taking that page to amazing 25KB of total data).
> Because this is coming from nim and nim design goals are to be efficient and elegant I can not help but notice the excessive bloat. The main forum page is 4MB of data (~700kb compressed) to basically communicate 3KB of information (plus avatar images which should not be more than few kb total anyway).
As a basis of comparison, HN's home is 43k, and old.reddit.com is currently 3.16MB including all assets, the sole markup is 363K (gzip'd to 104).
Most of the 4.2MB of data is cacheable; only ~128KB gets re-downloaded on subsequent visits (about 100KB of which has to do with Google recaptcha). With compression, what actually gets downloaded on subsequent vists is on the order of ~60KB.
That being said, I agree that the web is bloated and needs to go on a diet. We just need to be careful on where to spend our energy for optimization (caches are great and we should rely on them).
In principle I do not share that view. The browser does not start with this data cached. You actually need to load all of it. On a slow connection you may give up and never come again to reach a cached-load scenario. It is a lose-lose scenario. IMO cache should be used to make an already fast page lightning fast.
Web perfomance optimization tells me a lot about a product and its priorities. I do not mind bloated products that perform a certain function well for their users and never try to pretend to be optimial from an engineering standpoint.
But this is nim, language that markets itself as efficient and elegant, and execution like this can hurt their credibility and brand perception in the very community they are trying to market themselves in (developers).
I hope this doesn't come across badly but: NimForum is an open source project, as such I would encourage you or anyone to help improve its efficiency. I think there is a lot of low hanging fruit here and I'd happily help with any patches that fix them.
Back in 2018 when I implemented the basis for this modern reimplementation of the Nim forum I focused on the functionality and a solid CI so that we can easily accept others' contributions without fear of regressions. Unfortunately I had only 2 solid weeks to make this a reality and optimising the code size didn't make the cut.
It could, but it doesn't. As many have remarked already, it appears there is a bug that's causing the site to reload Google's recaptcha multiple times.
This is a big low hanging fruit fix, another is simply minifying the Forum's JS, there is also bound to be a lot that can be done on the Nim side to reduce the code size generated.
As far as I can see you already have a minification task [1] in your nimble file. May I ask you why you are not using it to save bandwidth both for your sever and your visitors?
And about 3.7MB is JavaScript (mostly reCaptcha—eww! why load it when I’m just browsing the site?—being loaded multiple times—quintuple eww!; the forum’s own JS is only about a megabyte). Just parsing and compiling that code (not even executing it!) will take over a second on many common low-end mobile devices still in use (even some new), several seconds on slower devices.
Caches hang or crash my browser. Please don't. (Amazon Kindle Paperwhite – yes, this has been my only web browser several times – no, I will not replace it with another Kindle when it breaks)
3MB of the 4MB data appears to be JS, and a lot of that repeated calls to recaptcha_en.js, which would be cached after the first call. Though I don't know if all those repeated calls are really needed.
Discourse is great software and makes it so much easier to launch a forum but i personally never enjoyed the interface. Maybe I’m nostalgic, but I prefer the old style *BB interface with explicit categories/sections. Discourse kind of blurs the line with their “stream” style. This looks like an interesting alternative but unfortunately has a similar interface.
Discourse has always felt to me like standing on a riverbank and watching random things float by. It mostly works for comments on a blog post, but when I need to unearth specific information or want to participate in a discussion that spans weeks or years, I long for a more traditional forum. When faced with it as a tech support channel, I usually end up frustrated.
That blurred line is what turns me off from using Discourse-based forums. It's a neat idea, but it's too, as you say, “stream” styled.
The way I envision it is chat apps are yelling a question in a café, Discourse is walking into said café and setting down a sign then walking out, while classic forums are pinning a note in a library corkboard.
I wish they would skip unnecessary gamification notifications that make interacting on it tiresome. If you use different discourse forums this noise adds up quickly and starts being irritating.
Those are not even gamification items...It fails to do whatever it was designed to do...Yaaay your first like...Wowww you replied 5 times...Like, who cares?
There is an option named `Skip new user onboarding tips and badges` in preferences though...
That was my initial thought too, but after awhile I think it wasn't so much about the flow of conversation I have problem with, but its default design, style, spacing and layout.
And so I decide to check out Discourse again, turns out they Announced [1] a $20m Series A Investment by Pace Capital and First Round Capital. Congrats.
1- There is excessive white-space everywhere (like every other modern web-app). I feel like it was designed as a mobile-first solution.
2- There could have been smarter features regarding gamification and karma. Currently you can only hit like button (the total like count of a user is not even visible)
3- If a post has many replies, it only shows them up until a certain point and you have to scroll down to view more. These javascript shenanigans should not exist in a forum app IMO.
I have never used a nimforum-based forum but a quick browse in https://forum.nim-lang.org/ shows that their UI is more dense (less white-space) which addresses point 1 above.
> the total like count of a user is not even visible
That’s wonderful. Definitely a fantastic decision to help prevent toxic conversations. So glad that “feature” doesn’t exist on Hacker News or in Discourse.
But it does exist on HN. I can see you karma and after a while, the upvote count is visible.
When I ask a technical question on a forum, I'd like to know if a user who is deemed helpful in the community (by like count or `solved` count) is replying to me.
> 1- There is excessive white-space everywhere (like every other modern web-app). I feel like it was designed as a mobile-first solution.
So many programmers have this obsession with interfaces having every available piece of information displayed all at once. I just don't understand the appeal. I find these sorts of interfaces confusing and overwhelming, I literally can't think straight when I look at them. I immediately forget what I was doing due to all the distractions.
It highly depends on the use case. When people interact with a piece of software all day every day, they tend to appreciate having more information/actions available at a glance. Simple interfaces are better for casual users, but they generally mean more clicking around and tend to be slower to use for experts.
What annoys me most about Discourse are the following things:
No JS no joy. You see a completely white page, because none of it works without JS, not even an indication can be seen, that JS is required and it is waaay off of anything having a non JS fallback.
The above also means, that any knowledge in there will not be found via search engines. I have had it happen, that there was a forum of some community, and I did not even know it existed, even though I searched loads of stuff about the project. It simply never appeared in any search result ever. It is not discoverable.
Aside from not being discoverable, I find that any traditional forum software works better than discourse. For example I cannot properly see when exactly a reply was written, because it abstacts from a simple timestamp and shows only a month and year for example.
You are thinking of Nim with the white page; Discourse serves up a page with JS disabled. Try it! And of course everything is searchable. We would literally go out of business if we sold a web product that didn't index in Google.
As for the timestamp, click or hover over the timestamp to see the full date/time, etc.
> Discourse serves up a page with JS disabled. Try it!
The only content is in a <noscript> tag, so this complaint is probably coming from people only blocking third party JS.
When reading a random Discourse forum I don't intend to participate in, blocking all JS usually results in a smoother experience than accepting scripts from discourse-cdn.com. The only exception I've found is for massive threads, because there's no JS-free search.
Don't take this the wrong way: Being indexed by Google does not add anything for me personally, as I am not using Google. If there is a forum, which can only currently be indexed by Google, then I might even see that as harmful to the web in general, as it adds incentive to users to use Google, instead of other search engines.
I have experienced it multiple times, that stuff is only in some discourse forum and I see a blank white page. Basically a JS-wall. Not sure, if the people, who hosted it did something weird to cause that issue, but that is the impression I have of discourse forums. Rarely I allow its scripts to run and make the effort to view some content in any such forum. Usually I just close the tab.
Edit: Now I know why I might have had that experience: I also block loads of third party CDNs. Perhaps they were trying to load scripts and resources from there and that's why nothing was shown.
Edit2: But you are also correct: The Nim one does not show anything without JS.
I disable JS by default and had enabled JS for a couple of Discourse sites I read, but once it started working without JS I gladly disabled JS again because I find the consumption experience quite a bit better sans-JS: faster and more consistent with how the rest of the web works.
Discourse often shows a blank page with certain ad-blockers. JS is enabled, but the Discourse CDN gets blocked, so nothing displays unless the ad-blocking setting is overridden for that site. Some sites do load HTML, but others are blank.
With javascript disabled on mobile, you just get a header with links that don't work and "best viewed with JavaScript enabled" even on the official https://try.discourse.org/ site
What I don't understand is why discourse can't serve up content that isn't interactive. I get lazy loading if there are >1000 comments on a post, but for anything else, static content should be perfectly fine. Even when you have js on, ctrlf doesn't work because of dynamic loading so the site has to hijack the shortcut with different interaction patterns (pressing enter doesn't go to next result, it loads a separate page).
I just tested with the 999 comment post on that site. First page load with no cache is 5.98MB (1.25MB) and took 5.76s load time. Holding the end key to get to the bottom until I hit the last post downloads a total of 1.91MB (718KB transfer) and takes 70s! That's including all the pictures. That cannot be better than than loading all text and optionally deferring all img loads, right?
"Lets not load so much static content all at once, it will be slow!", but then serve megabytes of JS dependencies and perhaps some web fonts for good measure. Unfortunately this is becoming more common.
> I find that any traditional forum software works better than discourse. For example I cannot properly see when exactly a reply was written, because it abstacts from a simple timestamp and shows only a month and year for example.
Yes, I agree. I like the FluxBB based forum that Arch community uses and find the look & feel a good balance between excessive use of whitespace and overly cramped UIs.
> No JS no joy. You see a completely white page, because none of it works without JS, not even an indication can be seen, that JS is required and it is waaay off of anything having a non JS fallback.
I am not sure if this is configurable but the official kotlin forum [1] uses discourse and I can see the content when js is disabled. A lot of interactivity is disabled but the footer does have a clear message that the site is best viewed with js enabled.
I have often found kotlin forum posts through google search, so there doesn't seem to be any issue on the SEO front.
Nim has been using this forum for years. It's been very solid as far as I can tell.
In case there are any programmers in this comment thread, more interesting to me than the amount of whitespace or how similar the interface is to Discourse, is the source code. It's a great example of the power of the Nim language for application programming.
FluxBB [1], the PHP based forum that powers the Archlinux wiki works fine without JS, and IMHO looks fine too.
If you are looking for something minimal with threading support, fossil scm [2] includes a forum. You can checkout the forum for fossil itself which is powered by fossil.
Thank you! Indeed there are no tests. I'm not familiar with testing in Django or web apps in general but I want to add some once I figure out what and how I should be testing.
Well, in my limited experience, the best way to do testing is to start with tests first (TDD/BDD) - and the second best, when you've started, and have no tests - is to start from a bug: someone reports a real bug, in your real system - and you try and write a test that's failing in such a way, that when you alter the system to fix the bug, the test will pass.
Typically the first test will be quite painful - you might have to install a testing dependency, possibly re-work some code to allow for testing etc. But then, the next test should be a little easier. And after a few bugs and tests, you might find that for your next feature, you can somewhat easily start with a failing test.
I am loving this, thanks for sharing. My only request would be for a publicly hosted image so I can easily selfhost for my own little campfire community.
Hmm I might add one thanks for the suggestion. I don't use containers for this since it's just python WSGI on apache2 (no database image required since it's sqlite3)
Thank goodness. I've always been confused by Discourse. Discourse was supposed to be the modern answer to all the homegrown PHP forum solutions of the previous decade. Instead, it just ended up being yet another bloated JavaScript web app. And given who the creator was, especially disappointing.
Are you speaking from a developer or user perspective? As an user, I've always found Discourse to exceptionally user friendly and fluid to use. I find it a chore to navigate a forum that is not using Discourse.
The fact that discourse hijacks ctrl-f drives me insane. Sometimes I want to jump to a particular part of a thread featuring a word quickly, but discourse gets in the way.
And, though this is more subjective, I find the UI confusing. I end up browsing the github issues or even reddit for answers to questions just to avoid discourse on projects that use it (discuss.linuxcontainers.org - I'm looking at you).
Discourse doesn't load the whole thread, it's loaded on demand (which is a reasonable choice for many enormous threads), so "quickly" would instead be "not at all".
which is why phpbb or even (god forbid) reddit is more ergonomic to use for me. If the thread is enormous, then split it into pages. And if it's not enormous, the discourse solution ends up being heavier and slower than just loading the whole discussion at once.
Discourse's voodoo ends up being more annoying to me than useful, not unlike infinite scrolling.
(obviously this is all subjective..I can only speak to the fact that I get annoyed by it and thus instinctively avoid discourse when I can)
You still have to go through multiple pages in phpbb and similar old forum in long threads. Same issue, usually slower than scrolling down bring the apparent end of page in discourse
> You still have to go through multiple pages in phpbb and similar old forum in long threads.
Except for long threads the start of the thread is usually irrelevant beyond the first page or so (if even that) so you can just jump to a few pages from the end and go from there. Good forum software remains comfortable to use at million-post threads, and will render 100-post pages in a small fraction of a second.
IME discourse's "infinite loading" nonsense is just slow and a mess. Go check the Rust discourse's crate of the week thread, it is a slow mess to navigate, the browser's scrollback gets hopelessly lost, and of course the in-page search is broken.
That's a thread of a thousand comments, it's quite literally nothing, and for all that it completely fucks up your history to boot.
... because it's not possible to load the whole thread at once, so the normal scroll bar will be jumping up and down as you scroll to load content on demand. And really, if you want to jump to a particular post in a long thread, the special scroll bar is easier than pages anyways because it's continuous.
Really, the scroll bar IS a fresh take on the page list, except it's actually useful for navigating the thread and doesn't force readers to deal with the useless concept of numbered pages when such arbitrary divisions are no longer the primary way of navigating the thread. Once you choose to not implement explicit pages (and accept that it's not possible to load the whole thread in one request), the thread scroller is a natural consequence.
Agreed. I was initially pretty turned off by the app-iness of Discourse, but after using it for awhile I've come around and actually enjoy it quite a bit now.
What's wrong with Discourse: (and other silly scroll-hijacking pages):
1. Scrollbar no longer works properly when you try to navigate thread up and down. Normally you make mental note where there is something, and you can go back to it. But in Discourse, it isn't there, because you scrolled down, it loaded more replies, now you go back and there's something else in there.
2. You slowly drag scrollbar with your mouse. You near end of the page... it loaded more content, and now because you're still slowly dragging scrollbar, it again warps to the end, and WTF just happened, where is the thing I was reading!? Everything jumps and you just skipped dozens of replies.
3. Click any link. There's useless loading spinner, but oh well. Click any link, internal or external, and then go back. WTF!? I pressed back button, page should be in the same position it was before! It randomly snapped to some post, up, down, whatever, but it is somewhere else and now again, where is the thing I was reading!?
I cannot understand how anybody thought this was good design. It is horrible. Besides issue 3 which presumably could be fixed, issues 1 and 2 are fundamental problems. You cannot have page that have dynamic height and at the same time have scrollbar that is remotely useful. If you really insist on dynamic loading (and I don't know why you would, plain old HTML page is faster anyway), you could preallocate page height so it will work properly, but then you're back to square one: you need pagination.
You’re using the browser scroll bar when reading the text of a page to scroll through the text?
I’ve never heard or noticed anybody doing this. You must go crazy on infinite scroll webpages.
I’m using the PageUp and PageDown keys for page wide navigation and the arrow keys for the lineheight movements. On a desktop machine I’m using the scroll wheel as well.
I would only use the scroll bar for pixel perfect positioning of the viewport.
> Would you rather have click + wait to refresh the whole page
Considering it'd be faster than what discourse does, yes.
> to see the next 10 post on that thread?
If your forum has unconfigurable 10-post pages I would suggest getting a better forum software? One decades-old forum I frequent loads its default 40-post pages in half the time it takes discourse to load a new section, and using half the data, despite having to reload the entire HTML page.
And it doesn't waste a third the screen on a virtual thread scrollbar.
It's an alternative to Discourse though, e.g. it's a SPA. Nothing I see in the description entices me in the least... and phpBB and others are still being developed, and still rock.
I tried out Nim earlier this year. Great language, but the forum was very frustrating to use.
Every page takes a few seconds to load. If you open a page in a new tab, you still need to wait for it to load when you switch to the tab. The forum was the most annoying thing about using the language.
huh, that's strange. I've never seen any page on the forum take longer than a few ms to load. Just navigated through a couple of threads and all appeared nearly instantenously. Can you share a bit more about your hardware/setup?
I definitely see a discernible delay, on some pages more than others. Measuring on a heavier page (https://forum.nim-lang.org/t/1207) produced a result of more than seven seconds for a full page load, but to full readable load / DOM mostly complete (if I wasn't watching devtools, I would have figured it was fully loaded) was only around two seconds. Definitely not just a few ms, though.
Having all those globals in the code generated by the Nim compiler seems less than ideal, though. Especially since some of them have very generic names like "raiseException" and "addInt", which could conceivably conflict with some other JS library.
Yes, Nim being the library in this case. These names are not defined by the application but are part of the code that the Nim compiler generates for any application.
If you wanted to integrate some Nim code in an existing JS application and your existing application already defines the name "raiseException", or "unhandledException", or "framePtr", or "lastJSError", or ... that's going to be a problem.
This is still better than original Discourse which for this same thing needs 6MB of data (1.2MB compressed). However Discourse does not claim to be efficient or elegant.
This kind of web performance engineering attention to detail will go a long way. The benefits for users on slower connections are particularly important. And as an engineer, showing few KBs of information with few MBs of data should bother you. Your information to data ratio is less than 1%. Excessive reliance on JS does not help. The best way to think about JS is as a feature enhancer not a feature requirement.
See also: https://idlewords.com/talks/website_obesity.htm
Having said that I think it is absolutely great that you are doing this and I hope we can get efficient and elegant forum software. This will certainly help nim adoption if executed in the way that is true to your values.
Edit: A competitor has been mentioned in comments:
https://fluxbb.org/forums/index.php
They are showing similar amount of information in "only" 100KB of total data. And 80% of it is CSS (which does seem excessive for what it does, I am sure 5KB of CSS would do the same job taking that page to amazing 25KB of total data).