The title, as it stands, is trite and wrong. More about that a little later. The article on the other hand is a pleasant read.
Topology is whatever little structure that remains in geometry after you throwaway distances, angles, orientations and all sorts of non tearing stretchings. It's that bare minimum that still remains valid after such violent deformations.
While notion of topology is definitely useful in machine learning, -- scale, distance, angles etc., all usually provide lots of essential information about the data.
If you want to distinguish between a tabby cat and a tiger it would be an act of stupidity to ignore scale.
Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations. That happens, but to claim deep learning is applied topology is absurd, almost stupid.
> Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations
But...you can't. The input data lives on a manifold that you cannot 'trust'. It doesn't mean anything apriori that an image of a coca-cola can and an image of a stopsign live close to each other in pixel space. The neural network applies all of those violent transformations you are talking about
Only in a desperate sales pitch or a desparate research grants. There are of course some situations were certain measurements are untrustworthy, but to claim that is the common case is very snake oily.
When certain measurements become untrustworthy, that it does so only because of some unknown smooth transformation, is not very frequent (this is what purely topological methods will deal with). Random noise will also do that for you.
Not disputing the fact that sometimes metrics cannot be trusted entirely, but to go to a topological approach seems extreme. One should use as much of the relevant non-topological information as possible.
As the hackneyed example goes a topological methods would not be able to distinguish between a cup and a donut. For that you would need to trust non-topological features such as distances and angles. Deep learning methods can indeed differentiate between cop-nip and coffee mugs.
BTW I am completely on-board with the idea that data often looks as if it has been sampled from an unknown, potentially smooth, possibly non-Euclidean manifold and then corrupted by noise.
In such cases recovering that manifold from noisy data is a very worthy cause.
In fact that is what most of your blogpost is about. But that's differential geometry and manifolds, they have structure far richer than a topology. For example they may have tangent planes, a Reimann metric or a symplectic form etc. A topological method would throw all of that away and focus on topology.
I don't think that was their point, I think their point was that neural networks 'create' their optimization space by using lengths, distances, and angles. You can't reframe it from a topological standpoint, otherwise optimization spaces of some similar neural networks on similar problems would topologically comparable, which is not true.
Well, sorta. There is some evidence to suggest that neural networks learn 'universal' features (cf Anthropic's circuits thread). But I'll openly admit to being out of my depth here, and maybe I just don't understand OPs point
once you get into the nitty gritty, a lot of things that wouldn't matter if it were pure topology, do, like number of layers all the way to quantization/fp resolution
The word "topology" has a legitimate dictionary definition, that has none of the requirements that you're asserting. I think what you're missing is that it has two definitions.
In blog posts about specialised and technical topics it is expected that in-domain technical keywords that have long established definitions and meanings be used in the same technical sense. Otherwise it can become quite confusing. Gravity means gravity when we are talking Newtonian mechanics. Similarly, in math and ML 'topology' has a specific meaning.
The word "topology" is quite commonly used in all kinds of books, papers, and technical materials any time they're discussing geometric characteristics of surfaces. The term is probably used 1000000 times more commonly in this more generic way than it's ever used in the strict pedantic way you're asserting that it must.
Surfaces certainly have a topology (potentially more than one), surfaces are examples of one kind of a topological space, in fact the next interesting one after a curve. So I will not be surprised at all with co-occurrences of 'surface' and 'topology'. But surfaces and topologies mean different things.
Dogs have fur. Dogs are an example of a furry animal. But dogs and furs are not the same thing although they may appear in the same text often.
Topology is a traditional as well as an active branch of applied and pure mathematics, well, Physics too.
Math magazines for high schoolers have articles on it. Colleges offer multiple courses on it. Some of those courses would be mandatory for a degree in even undergrad mathematics.
If one wants to do graduate studies then one can do a Masters or a PhD in Topology, well in one of it's many branches.
It's also not a new kid on the block. It goes back to ... analysis situs ... further back to Leibniz, although it began to crystalize formally after Poincare.
If someone wants to use the phrase 'differential calculus' to mean something else in their love letters and sweet nothings, that's absolutely fine :) but in Maths (and Machine Learning, well, with quality of peer reviewing this might soon be iffy) it has a well established and unambiguous meaning.
Note because of its shared beginning at the feet of Leibniz, comparing it with calculus is not an unfair comparison.
The most common uses of "topology", whenever used to convey a geometry-related idea, is in the more general sense meaning "surfaces". Only one out of a million times is anyone ever referring to the specific mathematical field of the same name to which you refer.
"Topology is a branch of mathematics concerned with geometric properties preserved under continuous deformation (stretching without tearing or gluing)"
That is indeed the established meaning of topology, more so in mathematics and the blog post was on applied mathematics. That it may mean something else in other contexts is irrelevant.
I rest my case.
> The most common uses of "topology", whenever used to convey a geometry-related idea, is in the more general sense meaning "surfaces"
Erm, citation please.
I included a search on Amazon on topology https://www.amazon.com/s?k=Topology&sprefix=topology+%2Caps%... (without even adding the keyword maths. None of the results seem to be about surfaces. Shouldn't there have been a few ? Wouldn't Amazon search results reflect the general sense meaning ?).
If it were true, wouldn't the Wikipedia pages have talked about that general sense meaning first ?
Alternatively, I would say, take a breath. Is this hill really the one worth dying on ? There are better ones. Have a good day and if work permits, get yourself a juicy topology book, it can be interesting, if presented well.
The question was never "Is topology a field of mathematics". The question was, is that term most often used to refer to surfaces in general, and the answer to that is still 'yes'.
Could you drop Amazon a message. They should really change the search results if that's what topology means in general. I am sure they would be delighted to receive the bug report.
For laughs, I asked ChatGPT to use 'topology' in place of 'surface'. Here's what it wrote:
The topology of the lake was so calm it reflected the mountains perfectly.
She wiped the kitchen topology clean after cooking dinner.
After years of silence, the truth began to topology.
The spacecraft landed safely on the topology of Mars.
A thin layer of dust had settled on the topology of the old table.
He barely scratched the topology of the topic in his presentation.
As the submarine ascended, it broke through the topology of the ocean.
The topology of the road was slick with ice.
Despite her calm topology, she was extremely nervous inside.
The paint bubbled and peeled off the topology due to the heat.
Our disagreement aside, I think these are hilarious. We should agree on that.
The best was
The surface of a doughnut resembles that of a coffee mug due to their similar structure.
The phrase "applied X" invokes the technical, scientific, or academic meaning of X. So for example, "applied chemistry" does not refer to one's experience on a dating app.
Topology is whatever little structure that remains in geometry after you throwaway distances, angles, orientations and all sorts of non tearing stretchings. It's that bare minimum that still remains valid after such violent deformations.
While notion of topology is definitely useful in machine learning, -- scale, distance, angles etc., all usually provide lots of essential information about the data.
If you want to distinguish between a tabby cat and a tiger it would be an act of stupidity to ignore scale.
Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations. That happens, but to claim deep learning is applied topology is absurd, almost stupid.