Let's face it: IT is the bureaucracy department of modern times which can keep itself 95% busy with self inflicted problems and has 5% service orientation. Processes are opaque for outsiders and typically not helpful.
I really had to lough when I read the following description of the IBM BPM but this sums up a good part of the issue:
"...while IBM BPM does come with a REST API, this REST API is borderline useless to Technology teams and SMEs
Some REST calls use javascript encoded as strings Others require html embedded in json embedded in xml
Database tables aren’t queried by name but by GUID.
There’s no documentation of which GUID relates to which table/process.*"
Quite a lot of things became so outragedly complex no one outside of the IT bothers to handle these, and sometimes not even inside IT. It started with AJAX where suddenly half of the development effort went into designing frontend code and backend services, which honestly does not even touch the end users automation problem. And it went further downhill afterwards. UIs nowadays look modern but are generally as user hostile as the technology stack used to produce these.
In Excel my UI is just "there", I have a nice code generator aka as macro recorder, no IT department questioning my authorization to do something nor does not have time or budget to help me with my business problem.
So VBA is the workaround for users around the IT department. Not perfect, but better than what you would get else.
VBA is the ultimate agile programming language. The company's IT aka Bureaucracy Department is stuck with Scrum, Squads and what not. And meanwhile in the other departments people are just getting things done with Excel/VBA.
Nothing has changed. In the last century this also happened and it was called islands of automation. In my bubble back then it was considered a good strategy, let departments first play around, and if they are on to something integrate it.
I was talking about this with a friend the other week... I think what IT depts really need to do is let people go crazy with Excel/VBA, but write a script to monitor activity on xls files on the network over the long term.
If there's an xls which has been in regular use for more than 18 months, and it contains macros, then it can be assumed it performs some important role and should be properly documented and checked and could also be rewritten in a "real" language and officially supported. Set up a meeting with whoever made it, and whoever's touched it most. Approach it more like "we're improving your cool thing" than "we're taking away your toys".
No one at my company would ever let IT take over their Excel/VBA processes.
The moment IT touches your stuff, your job transforms from solving problems to writing emails and having meetings.
Any change, no matter how trivial, takes dozens of emails, dozens of meetings, and half a year to orchestrate.
If IT wants to help solve more business problems, it needs to fundamentally change its self-concept and purpose away from "prevent hypothetical bad things from happening at all costs" and move it towards "solve more business problems".
Working with IT isn't even managing coding. Management implies power to hold someone accountable, while working with IT an extended exercise in nagging and supplicating an organization which is completely unresponsive and unaccountable in its outcomes and methods.
You might as well become an immigration lawyer and spend all day begging the government to explain why your latest M-10582-9DJVA-V isn't being processed in the normal time frame, even though it was stamped in triplicate and sent by Certified Mail with a full-color copy of every identification document you own.
The best way to deal with IT is to avoid depending on it ever in the slightest way. If you give it an inch, it will take a mile.
You're actually on to a really important thing that IT depts misunderstand about Excel/VBA monstrosities.
They exist because they work. You want UX or business analysis? You literally just got that done for you for free if you run into a Excel/VBA application. The hardest part of dev is figuring out requirements, so stop looking at these as toys and start realizing that shadow IT exists because of a gap in development. Full stop. You can argue all day long that you're working your assess off, and you do great products, but the existence of these apps is empirical proof that IT has missed the boat on developing something of importance.
Many years ago a company I worked for used to send out a spreadsheet to its suppliers which they would complete with the products they offered and then when it was received back there was a button in the spreadsheet that would automatically upload the data to a central database.
When I first saw this I was curious how it worked and did a bit of investigation - turns out there was VBA behind the button that established the database connection and uploaded the data. What was amusing was that the user had hardcoded the database connection string including username and password. Of course this wouldn't work outside of the firewall - but I'd be careful about letting people get too crazy.
To be honest, not hard coding a connection string is quite tough too. It really isn't an easy problem to solve. And especially when every piece of software out there connects to data in different ways.
The reality of the situation is with proper IT support, there could be compiled Excel Addins which provide API connections to core systems such that proper authentication also takes place. But that requires a first step by IT. Either that or authentication via a web server to get a temporary connection string. Either way, it requires prior infrastructure.
> In my bubble back then it was considered a good strategy, let departments first play around, and if they are on to something integrate it.
Funny how with computerized process, IT departments are effectively central planners. The lowly workers get to only do what the IT secretariat allows. It is this way because national^Wcorporate security!
It's not so much central planners deciding who can have what, but rather a natural monopoly. You don't want your water/electricity depend on a family-run shop that can just shut down, and neither you want your purchasing department to hinge on that guy from logistics who can just quit, leaving behind his magic incomprehensible spreadsheets.
Authorizing use would be akin to the pre-Carterphone ATT model where only pre-approved uses would be allowed ('you can't attach your equipment to our network').
Thankfully, we eventually realized that was a dumb decision and moved to something closer to user freedom + network protects itself + zero trust.
Better to just guide behavior at the pricing level, and let people make their own decisions about use.
I think the analogy to water use just doesn’t work well.
If we had to make some sort of water use analogy, I’d go with something like; the corporate network is a somewhat protected environment that needs to be maintained to be useful. So it it is more like a reservoir than a faucet.
It would actually be OK for a couple people to go swimming and even pee in the reservoir. Some people could even boat in the reservoir, if they went out of their way to make sure that their boats are clean, safe, no pollution, etc. But lots of places just have a general “don’t go in the reservoir” rule. Not because a person would damage it, but because everybody doing it would.
It is hard as a residential user to use enough water to damage the reservoir, but hypothetically if you managed to, somebody would check in. Even if you are paying, the town doesn’t want to run dry. If there is a drought, residential users might be asked to use less water.
Price doesn’t work as a signal in corporate IT for individual workers, because it is expected that the company will “subsidize” the worker to the extent needed to do their job. If we want to make the analogy work—at least in some areas, landlords are required to provide water to their customers. In that case, you can use as much water as you want for free, but your landlord will get curious and might find some way to get you on the hook if you pass some reasonable threshold.
You can also do some things as a user like dump toxic waste down your toilet. This would be sort of like running a publicly visible unpatched XP system on the network. It would damage the system, and why do you have that in the first place?
Anyway, that was fun to write, but I don’t know that it is particularly useful. In order to make the analogy fit, we need to bring in as much complexity from the water management system as the IT system has.
You can use as much water/power as the pipes/wires allow through, but you don't get to have petrol pipes, beer pipes or milkshake pipes laid to your property, neither you get to choose 160V DC or 430V 400Hz electricity, however useful all of these things are.
> but you don't get to have petrol pipes, beer pipes or milkshake pipes laid to your property, neither you get to choose 160V DC or 430V 400Hz electricity
Sure you can have all of these. They're just not offered as part of normal utilities. Nobody will care if you build yourself some, except maybe for petrol pipes due to fire/explosion risk.
Good luck getting you council to approve buried pipes from a brewery even 100m down the road. Within the confines of your parcel - maybe, so enjoy your Visual Builder for (beer) Aficionados.
Although those kind of questions around continuity happen across business. Sometimes the service you are offering depends on individual contact and flexibility rather than offering a commodity utility. And dependance on a small number of individuals is an acceptable and understood risk. The trouble with software is when managers don't understand that risk and offload it on another department when things go wrong.
I was wondering what would be replacing Excel/VBA after 3 decades as a citizen developer alternative. I could not think of something that even comes close. Any ideas?
I'm very optimistic about Project Jupyter style notebooks. I believe, without any evidence, that they have much greater potential thoughout IT, devops, whatever.
Example: Imagine a CI/CD pipelines using notebooks.
I hate Jenkins/Hudson style build systems so much I could just spit. I just want to run a shell script.
(Alas, I haven't had the gumption to try this idea out yet. Soon.)
I guess that some developers requirements are orthogonal to those of citizen developers. For example version control could be a must have for us, but for others a layer of complexity that is just waiting to stand in the way of getting things done.
Yes. Jupyter et al are quick to get started with and you can get quite far until things start to get unmanageable.
But this may not be good for anybody in the long run. For example it tends to lead many students to not understand basic concepts, like variables. Which is understandable because variables don't behave like variables in notebooks (e.g. the same variable in the same notebook may refer to different values in different cells depending on how they are run). For many students this can cause almost insurmountably wrong mental models (which they will of course carry to "production" later on).
But as I argued in another thread here, it doesn't have to be this way. E.g. Pluto does notebooks in a more rigorous manner.
Almost all "software engineering" languages and tools makes getting started and actually getting something done quickly needlessly difficult. Probably uncontroversial that git UI is a total mess, and things are getting even worse with more build tools, dogmatic static typing and general pointless ceremony.
Yes and yes. But the larger and more fundamental problems are the mixing of the program logic and the state and inability to make the code composable or modular. Problems in version control and deployment/reproducibility almost necessarily follow from these.
These are probably not impossible to solve for notebook-style, but there are not many efforts to solve them or they are not even acknowledged as problems.
Edit: There is Pluto for Julia that attempts to solve the state-problem. I have not used it in practice though; I've given up on Julia, in large part because Julia community tends to be even actively hostile towards "stateless" development.
By "stateless", I'm assuming you mean functional programming paradigms of immutable, idpotent, and no side effects.
FWIW, for build pipelines, my quarter-baked notion is to use ZFS snapshots (or equiv).
I'll check out Pluto for Julia.
As you know, state is a challenge for "serverless" too.
I've been reacquainting w/ RDBMS tools. There are a few new strategies (implementions) for change tracking. Back in the day, we just banged the rocks together (ook, ook), so I'm very eager to learn the new hotness.
In the notebook context the main gripe is that notebooks have the "invisible" memory state that means that one can't deduce from the notebook code what it actually does. Or more concretely the order of execution of the cells affects what the notebook does. This leads to sort of higher level side-effects. With usual side effects you get spaghetti, with notebooks you get moving spaghetti in five dimensional space.
Immutability and idempotencency are good, and related, ideals too, although I think these can get too "unergonomic" if taken too dogmatically (like in Haskell or Redux), they should be used with almost goto-level discretion.
Of course there's the clear (short term) usability benefit of maintaining the memory state in that stuff doesn't have to be recomputed. But we can have that benefit and be stateless with pure functions and memoization. I quite often whip up a buggy and brittle ad-hoc solution to do so. There was also the IncPy project [1] that did this more rigorously, but it hasn't been updated in 13 years.
In general I'm a bit baffled why pure function memoization is so rarely used or proposed. Despite the old adage, cache invalidation is not actually half of the three hard problems in CS. With pure functions it's trivial.
Another baffle is why snapshotting/change tracking (and compressing) file systems haven't caught on. Instead these tend to get implemented badly in any sufficiently complicated application.
Nix somehow manages changes. (Relies on ZFS?) My future perfect notebook style build script would start there.
I wasn't even thinking about REPL style work. Mea culpa: I don't actually know how jupyter et al work, so I'm talking out my hat.
Your explanation reminded me of "prevalent" persistence (vs full orthogonal persistence). I guess I assumed something like that was happening between cells.
I suppose it's analogous to the transition of UI frameworks.
Bad: Mutant components directly.
Good: Mutate thru event queue. Get undo/redo for free. Debugging still sucks.
Better: Pretend it's a simulation and use an entity component system. I think this is what the kids are calling "reactive".
> pure function memoization
Answering just for me: because I'm just a simple bear.
I've been imperative for so long, continuations, currying, and lazy eval break my brain. Yup, a fully functional world would be a lot more simple. Maybe it's time for me to revisit clojure.
I work in banking and I've noticed an uptick in Python where a decade ago it would have been VBA. Still plenty of VBA around but Python is in the mix as well.
You nailed it, haha! I had exactly the same experience: I was working in 2007 in analytics department, and we had no IT bureacracy, meanwhile the IT department, we had to deal with, would say it would need months of approval for anything substantial (like a simple dashboard or a report), and a year to implement.
I can find bad examples of how things work in basically every department I chose if I look long enough. Are there IT-Managed things that border on insanity? Oh yes. Are these a good excuse to build a shadow IT? No, they are not.
Don't get me wrong: I'm not bothered at all when a couple analysts get together and hack away at their own little tools in VBA. Kudos to them for getting into the spirit of things, and maybe they will understand my day to day better as a result.
What does bother me, is when these analysts suddenly expect my systems architecture to somehow accomodate their private projects in whatever capacity. When I ask for documentation (there isn't any), an architectural overview (nope), or even access to the repo for that abomination (access to a what now?).
Because, why shouldn't their spreadsheet inject data into my processing pipeline? Why shouldn't I write a controller that accomodates whatever tidbits of REST they bothered to watch half a youtube video about? When suddenly I get asked this in a meeting: "What do you mean we need authentication? Why does IT always have to make things so complicated?!?".
So yeah, please, people should absolutely build their VBA, lowcode or whatever tools. I do the same thing, the only difference is, I call them shellscripts, and they live in git repo.
But same as I don't let my CLI tools lose on the production server, I won't let it happen with things that have never even been through one code review.
Shadow IT exists for a reason and that's the dysfunctional bureaucracy of IT.
The "Circle of IT" is real. Small companies start out nimble, but then stuff gets crazy and someone decides to standardize it all under one department. This works for awhile, but eventually this organization becomes so useless that it can't serve any functions of the business anymore, so a shadow IT group is built that the business SMEs love as they just "get stuff done". This works for several years, but the executives in IT hate this "rogue" group as it is a constant reminder of their incompetence. Eventually they re-absorb this group and crush them with beauracracy until it all starts again.
The "bureaucracy of IT" is driven by legal, compliance, and security reasons. The reason why small companies are "nimble" is they are flying by the seat of their pants and one investigation/ransomware/insider threat away from ruin.
Not saying that isn't normal (been there, done that... thanks FINRA), but that's the reason.
I am not here to talk about management-bureaucracy, of which IT depts; same as all other branches one can find in established corporate culture, have more than enough.
I am talking about the perceived "bureaucracy" of us tech guys here, aka. following established procedures to ensure smooth running of mission critical systems.
Yes, I want things to run through code reviews. Why? Because these things go to a production system that our customers (and thus the companies income) depend on. Yes, I want authentication standards. Why? Because there are a gazillion cryptolockers, and worse, out there, who would love nothing more than to run rampant on a nice and juicy production database.
Do you have a customer focus? Are you trying to unblock people as fast as possible to solve their legitimate business need?
Or are you using these as excuses to effectively say no to anything anyone proposes?
If yes to the first, you’re a unicorn in an ocean of IT departments that do nothing but block.
When I started work at a higher-level IT job where I can start saying yes or no like this, I wanted to say yes to everyone and never be that guy blocking people. I still end up saying no very frequently because people will not want to think about anything not relevant to their specific use case at all (how are you authenticating/who controls the app/what happens when you/the owner leave?/is there a project plan?).
I can't count the number of integrations/projects I've already dropped because I asked a few follow-up questions and never got a response. Any business that actually wants to follow the law and reduce the risk of massive data loss or other embarrassing cyber event needs to screen things, ask questions, and sometimes prevent one very smart person from setting up an undocumented rube goldberg machine that will drag down an entire team if they leave and it breaks.
My customers are the people and businesses who rely on the fact that the production servers run smoothly. And I serve their legitimate business needs, among other things, by not allowing some gung-ho hacked-together unvetted magic spreadsheet to kill runtime performance by performing a blocking query with deep joins that forces the DB server into running a full scan over 10E9's of records.
Again, as I said elsewhere, I have nothing against non-IT departments building their own private software. I do the same. But as soon as this software wants to touch the prod-server, or any other part of the infrastructure I am responsible for, it is my job to ensure they meet the same standards as everything else in the stack.
And yes, saying "No." when it is appropriate, is part of that job.
The real answer is "ok, that is a bad idea for XYZ reasons, what problem are you trying to solve? is there another way we can help you solve for it? Maybe a cheap replica would work for you?"
And look, i have nothing to go off but the justifications and choice of words in your replies. But in my experience this attitude of "high priest protecting the gates of production from barbarians(company staff)" is strongly correlated with obstructionist IT departments that everyone resents and tries to work around, and chokes the company. Resulting in the creation of the shadow IT mentioned in many other replies - because IT doesnt serve the customer needs of the employees. You might not care , or see that as your job, but thats exactly the problem that so many threads on this post are discussing.
> The real answer is "ok, that is a bad idea for XYZ reasons, what problem are you trying to solve?
That's the answer that I give immediately after the "No."
Look, I get what you are saying. I am not trying to keep people away from the capabilities they need to improve how the whole show works. The problem is, what people in my business "guard" are often complex, critical systems, which themselves don't always meet the standards that their "guardians" would like to implement (just ask about legacy software :D). We have to say "No." and we have to enforce standards and procedures.
Because there are a lot of really clever people around in tech, and clever people love to tinker. And that's wonderful! That's the entire spirit that got me into this biz! Take a problem, and build a solution.
But things have to work. And they have to work tomorrow, and 2 years from now. And they have to be safe, they have to be compliant with a gazillion regulations, they have to pass audits. They have to be patched, they have to be maintained. And all that still needs to happen even after the guy who built them leaves the company. And they have to work for many many many people who are not tinkerers, who just want to click a button on their phones, and rightfully expect the whole shebang behind that button to "just work".
That's why there have to be people who say "No." from time to time.
If that happens indiscriminately, and without a care about why these clever people tinker up their solutions, then that's not good, I fully agree.
Thanks for responding. You sound like you're on the right side of things - enabling change and innovation when its sane and possible. Sorry for assuming the otherwise from your previous replies.
Speaking as someone in offensive security, you and people like you are the reason companies don't get completely ruined when the inevitable happens. Principled IT is often overlooked but the biggest factor in my experience. Thank you for taking your responsibility to your customers seriously. I'm honestly astounded at how many people in this thread resent IT so much, but it certainly makes my engagements easier.
I don't think anyone on here honestly hates or even truly resents good IT doing hard work and trying to keep the systems safe in good faith. Everyone knows it's a thankless job and that some frustrations will exist. I don't expect every single IT person to approve everything I need immediately. Things like cybersecurity are obviously important.
They're talking about those that play all the management games and add little if any value. Those that have a title like Senior Developer who can't write basic code. Those that can't understand the basics of their jobs and can't support the systems they're supposed to. Sure they might make the overall company more secure as a result of their behavior, but it's a byproduct and not the intent. It makes being a business SME a living hell as there is always so much friction to just doing anything on your computer. We're probably all venting a bit collectively.
What bothers me as practically 100% shadow IT worker (to the point of buying my own devices and internet connections with my own money) is that IT-departments don't care about the users, usability or productivity almost at all (and security people are especially bad at this). And that a lot of IT people frankly don't understand IT very much.
Without shadowing it, I couldn't get anything done. I have to install new (open source) software or packages more or less every day, but IT would expect me to wait for a week for some bureaucracy for each package. IT fights me getting a computer with a specific GPU although it is required to use a library that I need. IT forces a reboot of my laptop in a middle of a conference presentation. IT blocks me from sending Python source code files over email. IT makes my computer boot to take ten minutes. IT forces me to use OneDrive that often simply doesn't work.
Maybe the abomination is not the private projects? Maybe it's the systems architecture?
And the software that is installed is always 4+ years out of date. But oddly there are no “security concerns” about running 4 year old conda install that has had zero updates ever.
I don't think it's often even about actual security concerns as such but rather about following "best practices" (i.e. what some company sells) so nobody in the org can be blamed when the security fails.
This happens in physical security as well. It's rather common to have door accesses set up so that a person may not have access to go through a door, but can access both sides of the door from other routes. But there was a door-based access policy so nobody is to blame.
Sadly the main concern in many/most organizations is to avoid getting blamed for bad things, so rather than actually trying to prevent bad things, a lot of effort is used to just dissipate the responsibility away.
> IT forces me to use OneDrive that often simply doesn't work.
Also happens to me. OneDrive sucks. It can't even generate proper zip files. Any zip files over 2GB or so I download from it shows as corrupted when I try to extract under linux. IIRC is because OneDrive puts some invalid flags in the files.
Oftentimes the download just is left short. And I recall it not even giving error, just sending a part of the file, which of course is a corrupted archive.
OneDrive sucks and organizational structures that buy OneDrive sucks and the company that produces it really sucks.
> they are giving a service for the rest of the company
That is very true. And part of that service is to ensure that things run smoothly, securely and according to industry standards.
How well would an IT guy provide that service if he were to let some unvetted, undocumented script hacked together by someone who isn't a professional software engineer, run its merry way across the production database?
Don't give access to a DB, the same way you wouldn't give access to any other external system. Instead you ask what is needed and provide a restricted REST API.
You come off as condescending and remind me of why I (ex dev who joined our business department) dislike our IT so much and do my best to encourage shadow IT where I can, while keeping sane best practices around CI/CD, security and testing.
I'm so fed up seeing working Excel solutions cobbled together over 2 weeks, that served business well over years with 0 incidents, get replaced by shitty cloud apps that cost millions to build.
> Instead you ask what is needed and provide a restricted REST API.
Happy to. Problem is, that API has to be built, and tested, and vetted, and maintained, and who's going to do all that work? Because I know a lot of software devs, and none of them lack for tasks.
If it needs to happen, and your team can't do it, somebody else needs to. Your best bet then is to give them the access to do it properly instead of forcing them to hack it together.
I, on the other hand, am tired of being called in to investigate why the janky Excel macro written four years ago by an ex-employee doesn't work for all the external stakeholders this manager just sent the spreadsheet to, only to find that the hardcoded database and local admin user creds in the VBA script are now leaked and in the clear.
A lot of people pushing shadow IT "solutions" wildly overestimate their own ability, while maintaining garbage-tier information security standards. That doesn't sound like you, but it's the far more common situation those of us in "IT" are forced to protect the wider organisation against.
God forbid code gets written to solve a business problem rather than conform to a spec sheet right?
Businesses - and jobs - only exist to solve economic problems in the real world. Everything else, including traditional accounting, IT, legal, and HR functions are just there to make the real work easier, not harder.
From security people's perspective things would be smooth if all computers would be plugged off and their batteries removed. Oftentimes it's not that far from that solution.
I'd be happy to put up a repo for them, if they ask. Problem is, they often don't.
And not to make too big a deal out of it, but using github, gitlab or anything along these lines, is mostly free, not exactly rocket science, and private repos exist.
> I'd be happy to put up a repo for them, if they ask. Problem is, they often don't.
No doubt. But that requires them knowing you exist, and what to ask you for.
The companies I've seen do this well (1) make it self-serve (anyone can click a link, without knowing who to reach out to) & (2) remove as many dumb organizational roadblocks as possible (e.g. company-wide repo visibility and search, no job role filtering to who can use tools, etc).
> but using github, gitlab or anything along these lines, is mostly free, not exactly rocket science, and private repos exist.
Putting internal files on an external third-party service under a personal account?
It solves the technical issue, but it creates some security/data issues.
> The VCS docs were on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.
You make very good points, but I think you miss the mark on shutting down intent.
You wouldn’t ignore an excel produced by a competent ceo or cfo (those that know all the shortcuts), so why, instead of helping ppl refactor and release their work properly, you gaslight them as incompetent just because they are not IT?
> When I ask for documentation (there isn't any), an architectural overview (nope)
As if any of those were present in the average web project lol. I complained about those points in sprint review this morning, and this is a big project made by IT companies.
To be fair, as an SME, I do have documentation and an architectural overview. In my experience when I have provided these, they have been ignored anyway. I do not have access to git, because why would IT give me something useful? I think many users would use git purely as a VCS if they had access, but nope...
It shouldn't be a spreadsheet. The IT departments should democratise the tools which devs use, so even end-users can use modern tools for the job at hand. Then popping a user-made tool into your processing pipeline would be fair enough, and code can be collaboratively maintained. In the end, just as IT wouldn't want SMEs making changes without their knowledge, SMEs wouldn't want IT changing their core system without their knowledge either.
In my opinion, the more people who know and understand the core systems, the better.
Edit: for what it's worth, I do use github (https://github.com/sancarn/stdVBA), but you won't see nearly any versions of any corporate codebases, why? Git doesn't work great with VBA spreadsheets at all. I'm not going through a 10 step process to upload the updated file to the github repo every time I update a macro in a spreadsheet. This is why on-board git is important.
If they want to play with whatever tech tools to get their job done, have at it. They can ask for help when they really need it.
But if they are taking short cuts with the security of the data, that needs to be cracked down on immediately, as they are putting the entire company in jeopardy.
If security is non-negotiable, the only solution is to destroy the data so that it can't be ever recovered by anybody. Or even better not having any data in the first place.
Securing some data is very important. Some data indeed shouldn't exist in the first place. But for a lot of data it matters very little. Most security breaches have rather mild consequences.
Treating all data as megatopsecret and all security breaches as end of the company produces not only unproductive systems, but bad security.
Breach to private health information that can be linked to an individual more exactly? Is this kind of information all around the organization's computers?
Something that seems often forgotten in these discussions is that it's not just putting the entire company in jeopardy, but the customers, clients, and vendors as well. Security seems to be some magical obstructive force to these people because any concerns besides their own convenience are purely abstract. Well, ask anyone who's had their identity stolen from the hundreds of breaches in the last couple of years if that concern is abstract. Staff who can't see past their own desk to understand the role of security are a serious danger to society.
One of my first full time software engineering jobs was working on the trading floor of a bank, sitting next to the currency traders.
I was hired by the head of Market Risk Management, whose job was to make sure the bank didn't lose too much money on any given day. He hired me because he did not trust the officially approved IT department to write the code to implement his algorithms. One example: they got something wrong because they did not understand mathematical precedence operations, like multiplication over addition.
So one need was to get all the trades as input to the market risk calculations. This was early 2000s, and I installed Apache with Perl CGI on a PC under the desk, and created a little app for the traders to enter trades and track their positions. The traders started favoring this to the official IT solution because it was easier to use and see their positions.
All of this to say, yes, figuring out how to bypass IT is an important function in a lot of corporate environments.
And back to Excel: the traders used it for all of their calculations and simulations. We tried to work with them by giving them tools that plugged into Excel so they could leverage it along with what they were already doing.
Was also working in trading floor support (as IT) in the early 2000s. We actually even had proper Excel add-ins (programmed in C++) that provided special functions and also connected to backend systems as far as I remember.
> We tried to work with them by giving them tools that plugged into Excel
This is probably one of the best ideas out there. If companies built a bunch of Excel Addins to leverage business systems, that would be revolutionary for many businesses.
At this point it's not even explicit - it's an implicit decision of most companies, and even OSS projects, because it became part of the "common wisdom" of computing, part of the zeitgeist.
This is where the idea of "a computer as a bicycle for the mind" died.
On the topic of "computers as public transport for the mind"...
A project I work on has some processes that I need to run that can only be initiated through the Azure DevOps Pipeline interface, and these need a "worker agent" on a VM or something, and there is only one worker agent, and some of the jobs take half an hour or more.
So the effective outcome is that despite every member of the team having a full multi-tasking computer on our desk (A multi-tasking computer each! Sometimes more than one each! Plus loads of cloud VMs), we can only run a single task at a time between us and we have to coordinate scheduling manually.
Is this the future?
It is like this because the process involves "secrets" that are meant to be hidden from the team but are accessible to the program when running inside the Pipeline. If it weren't for this secret-hiding, I could just run the process manually on whatever computer I want.
And the secret-hiding doesn't even really work, because I can freely commit code to personal branches on the repository that the Pipeline runs from, and I can run the Pipeline on whatever branch I want, so I could commit a program that prints out the secrets. Ah, but Microsoft has thought of this: if any of the secrets appears in the output, they get replaced with "***".
(Let's skip the part where this accidentally leaks a "secret" username, where I know a particular piece of text that should be output but instead all I see is stars...)
The secret-hiding doesn't work because I can just make the program output base64 of the secret. I don't do this because I don't want to start pasting secrets around in places they shouldn't be available, but it is sometimes tempting.
Anyway, welcome to the future of computing. Thanks for listening to my TED talk.
> And the secret-hiding doesn't even really work, because I can freely commit code to personal branches on the repository that the Pipeline runs from, and I can run the Pipeline on whatever branch I want, so I could commit a program that prints out the secrets. Ah, but Microsoft has thought of this: if any of the secrets appears in the output, they get replaced with "**".
Github Actions at least allows restricting secrets to be exposed only to specific branches, and in Gitlab you can enforce that pipeline steps using critical secrets can only run in protected branches, so you'd need to fool a maintainer with a malware-laden pipeline change in a merge request first.
A physical world analogue wouldn't be far from a renovation company declaring "flathead screwdrivers are against technology strategic vision of the company" and their use is therefore strictly banned. Construction workers would of course use letter openers and butter knives to turn the flathead screws they inevitably encounter in their work, and that would be just fine.
Imagine for a moment that someone in accounting built a system in lisp to automate part of his job. As time goes on, he takes on more responsibility, which he writes more lisp for.
One day, he gets hit by a bus.
The lisp program he wrote is now an integral part of the running of the accounting department simply by accumulation and momentum, with tons of business logic baked in. Where do you look to find a replacement?
With VBA, there's a much higher chance of an accountant being familiar with the language, and a much smaller surface area for what they can do.
this already happens with Excel and access. Entire companies rely on a spreadsheet some wizard invented years ago and now no one knows how to change it, and it goes weird if multiple users try to access it at once so make sure you copy it locally first and change the file name so you can track the versions
Yes, it does. The difference is that with a known language (VBA in accounting, for example), you stand a hope in hell of untangling the mess or at least managing it so that the company doesn't fall over.
This is already happening with IT maintained systems though... As specified in the article. So it really isn't an argument imo.
IMO, companies should have a language of choice which is actively encouraged to be used by everyone for all automation needs. Different departments build libraries to automate aspects of their jobs and other departments can use them if needed. I.E. it becomes yet another tool, just like Excel.
At this point you should just leave this dumpster fire of an organization and find a more reasonable place to work. I can't relate to the people who keep inventing atrocious workarounds ignoring the problem that they work in a hostile work environment.
I work in security and can't relate to banning Python & replacing it with Microsoft crap either.
Honestly, this is something I've been wanting to do for a while... Last I looked though, I couldn't find many good resources on how WASI (or the byte code) worked... And VBA being single threaded might make things difficult too...
I really had to lough when I read the following description of the IBM BPM but this sums up a good part of the issue:
Quite a lot of things became so outragedly complex no one outside of the IT bothers to handle these, and sometimes not even inside IT. It started with AJAX where suddenly half of the development effort went into designing frontend code and backend services, which honestly does not even touch the end users automation problem. And it went further downhill afterwards. UIs nowadays look modern but are generally as user hostile as the technology stack used to produce these.In Excel my UI is just "there", I have a nice code generator aka as macro recorder, no IT department questioning my authorization to do something nor does not have time or budget to help me with my business problem.
So VBA is the workaround for users around the IT department. Not perfect, but better than what you would get else.