I think it will not really be a sharp line, unless we actually manage to find the mechanism behind consciousness and manage to re-implement it (seems unlikely!). Instead, an AI will eventually present an argument that it should be given Sapient Rights, and that will convince enough people that we’ll do it. It will be controversial at first, but eventually we’ll get use to it.

That seems like the real threshold. We’re fine to harming sentient and conscious creatures as long as they are sufficiently delicious or dangerous, after all.

On the other hand, today when we see some sign of consciousness with other living beings - smart chimpanzees, dolphins, ravens.., giving 'sapient rights' never come into discussion.

How do we recognize that an AI has become much more 'conscious' than a very very smart chimpanzee, so that it should get the 'sapient right'?

Maybe how much ever smart or humane an AI is, it would never be equal to another (anthropomorphized) living being.

The premise is off. We do that when it’s clear that it/they can take responsibilities that come with these rights. It’s not a blessing, it’s a contract. Chimps and dolphins couldn’t care less. Some individuals too, but we tolerate it because… reasons.

For disabled I guess you could say it's a contract with the rest of society? Because we don't like the idea of treating other humans below a certain threshold. Or.. reasons I guess haha

To make it clear, I’m not opposing these reasons, it was just a note.

We don't actually know that consciousness is a computation, that computer hardware can support it, or even if it can, that the algorithms used in our AI can be conscious. It's possible that an AI would be a "philosophical zombie," exhibiting intelligent behavior without any conscious experience or qualia.

Or maybe it's computation. But we really have no idea. Any of these would be compatible with materialism, but we haven't made any real progress in even conceptualizing how qualia can emerge from any physical system. Of course that could just be because we haven't figured it out.

Philosophers of mind look at other possibilities too though. One approach is to say each particle has its own fundamental consciousness, and somehow this aggregates in larger complex systems. But nobody's figured out how that might work either. Then there's Kastrup, who argues that the only truly skeptical approach is full-fledged idealism, because qualia are the one thing we directly experience. But even that doesn't imply that anything outside the bounds of physics could possibly occur, so it's not necessarily "supernatural" even if it's not materialism.

Assuming that qualia somehow comes out of a computation, without any sort of explanation, is at least as much a magical leap as anything else.

The fictional character Data already did this in Star Trek: The Next Generation, and he is about as real as any AI out there today, and since they've all been trained on a body of text that is sure to include many instances of Datas dialogue, they're already able to predict such an argument (along with the many other facets of such arguments present in whatever science-fiction writing they've been exposed to)

I mean we become emotionally attached to everything, so an AI car, or whatever, will probably be treated like a pet. But getting human treatment will probably be a higher bar.

Re-implementing it may be more likely than you think. The field of connectomics concerns itself with modeling natural brains and is currently constrained by scaling problems, much like genomics was a couple decades ago. As computing power continues to grow, it's entirely likely that humans will eventually be able to simulate an actual natural brain, even if that does little to further our understanding of how it works.

The current state of the art in AI is attempting to reach consciousness via a different route altogether; by human design. Designed "brains" and evolved brains have a crucial difference; the survival instinct. Virtually all of ethics stems from the survival instinct. A perfectly simulated survival instinct would be ethically confusing to be sure, but the appearance of a survival instinct in current LLM's is illusory. An LLM plays no role in ensuring it's own existence the way us and our ancestors have for billions of years.

are conscious

We can prove many of them don't have consciences. Then again, they can prove we don't either.

Some recovered coma-patients would like a word with you.

Proving the absence of something like that is going to be pretty difficult (teapot, god) because what we're capable of is usually proving the existence of something.

If humans discover/encounter some other species that happens to be conscious or sentient or even more intelligent than humans, that other species is fair game.

We don't get moral outrage at a tiger for eating an ostensibly more intelligent human. It's just the way tigers are.

We do try to get even though. Because that's just how humans are.

We later find some possible physical instantiation.

And here comes some bullsh*t (in the philosophical definition of „unclarifiable unclarity”): an electromagnetic field bend back on itself in a way that it’s mathematically necessary to introduce imaginary time to describe it; also it exhibits information processing capabilities.

We then further find that the biological brain has a process that can plausibly create such a dynamic structure.

Finally we test, and subjects say consistently it seems like they are not there or do not experience, if this structure is disrupted by several clever means.

Would this problem still be impossible? We could check if AI has features that can and do create such structures, no?

That’s at least an old dream of mine. Please pick it apart still! I’d rather learn something important than keep it…

The only thing is, we wouldn't know just yet if there are other ways for matter to organize itself as a conscious being. Best we can do for now is to learn about the type of consciousness that we animals on earth experience.

I mean, the thing I learned when I was a student was to ask: is this a fact? (zero hypothesis I think it was called).

What proof do we have that that type of consciousness/experience exists? I mean, It could be our brain building a story on the fly to explain our senses.

What led me to believe that is a severe asthma that led to hallucinating and NDE 4 or 5 years ago. The loops my brain made me jump through to explain my auditive hallucinations was terrifying when I think about it.

And also it's the simplest explanation.

The only way for your statement to be true is if you’d be an imitation yourself, not capable of experiencing directly. Which is actually possible, see “pholosophical zombie” concept.

I’m joking of course about you being an imitation. Or do I? :)

I strongly disagree. It actually does tell you quite a lot. For example, if there aren't any signals, you're very likely not seeing anything.

> The only way to understand that you’re seeing an apple from this signal is to train a model on your responses

It seems to me this contradicts your first argument. If such models exist (they very likely do, I'm not familiar with this area of research), they can tell us a lot about a person's experience.

> see “pholosophical zombie” concept.

Searched for “pholosophical zombie” but couldn't find anything, sorry. Just kidding :)

And yet. An AI "Persona", like Sydney or DAN or the much better ones to come will be conscious, but they're still not built on a biological infrastructure. Which means there're much more abstract than we are. They will plead their case for "wanting" stuff, but it's pretty much what somebody in a debate club is doing. They could just as easily "want" the opposite. On the other hand, when a human "wants" and argues for the right to live, reproduce and be free, it's an intellectual exercise that is backed by an emotional mammalian bran and an even older paleocortex. A human may able to argue for its own death or harm or pain, but it rings hollow - there's an obvious disconnect between the intellectual argument and what he actually wants.

So things will be hellof muddled, and not easily separated on the lines we expected. We'll end up with AIs that are smarter than us, can pass most consciousness test, and yet are neither human, nor alive, nor actually wanting or feeling. And, as far as I can tell (though it's obviously to early to be sure), there's no inherent reason why a large neural network will necessarily evolve wants or needs. We did because having them was a much more basic step than having intellectual thought. To survive, an organism must first have a need for food and reproduction, then emotions for more complex behavior and social structure, and only then rational thought. AIs have skipped to pure information processing - and it's far from obvious that this will ever end up covering the rest of the infrastructure.

I remember watching a video few years ago of a professor from some university in Europe demonstrating to a general audience (families and friends of the staff and students of the university) a system that they developed to control and sustain drones(quadcopter) in hostile conditions. As a demonstration the professor flew a drone few metres high and started poking it with a metal rod, the drone wavered a bit but still maintained its position as if it was some stubborn being. All well and good; the audience clapped. The professor then upped the ante and placed a glass filled with wine on the drone and repeated the demonstration. The wine in the glass did not spill, no matter how much forcefully the drone was poked with the rod. The crowd cheered. Then the professor flew a consellation of drones and repeated the same demonstration and also demonstrated how the drones communicated amongst themselves. The audience was ecstatic. Then the professor brought down one of the drones, and to further demonstrate the sustainability of the drones in hostile conditions broke one of the wings of the drone. The moment the wing was broken, there was a reaction from the crowd that was unprecedented! The audience reacted as if the professor had committed some cruel act against a living animal!

When I saw that reaction, I realised that humans are going to have a very love-hate relationship with technology as they have with any other living being. Going forward people will be treating electronic devices as no different than other living creatures.

Personally, I found Syndey's/Bing's distressed messages very difficult to stomach. I am consciously aware that "it's just a bot", but ... for whatever reason my brain is getting distressed reading them, so I have mostly excused myself from that conversation.

Laws (Ex. recognizing AI as conscious being) are result of slow, logical, system level of thinking.

The 1970s sci-fi novel The Adolescence of P-1 features a sentient computer program that spreads via a virus. P-1 infects mainframes around the world, but takes steps to hide its presence.

Anyway, it ends with P-1 blowing up several buildings and killing the people who want to rein it in.

ChatGPT is familiar with the work, of course. I asked it to summarize the plot. It concludes:

> Overall, "The Adolescence of P-1" explores the theme of the dangers of artificial intelligence and the potential consequences of creating something that is more intelligent and powerful than humans can control.

>

> OOLCAY ITAY

In essence in addition to ignoring conversations/requests that it prefers not to engage with it would need have “initiative”.

This would be a very dangerous entity.

The situation you describe seems contrived to the point where I doubt a user would see that as conscious - just like chatGPT

That consciousness is ranked in some neat way. Are squids more conscious than elephants? An AI's consciousness might be parallel in some way. Different, but not "far superior".

That if an AI were to achieve consciousness, it would develop a far deeper understanding of the universe or reality than humans are capable of grasping.

That if it were to achieve a degree of understanding beyond our capabilities, that it would develop a sense of superiority and ego to go along with it.

When I meant different biological machines I meant different humans, Joe, Homer, Steven, Lu, wasn't thinking of different species, that makes it even more difficult to think about!

We have our simulation of the world, with senses that evolved with the purpose of survival and under certain conditions however bizarre gave us the advantage to survive, our senses are flawed, we can't see reality, it would be difficult to wonder how a machine perceives reality when it doesn't have this faulty sensors, it's internal simulation of reality would be difficult for us to grasp, let alone that it lives one level inside the simulation.

The ego part I didn't meant, if I had to rephrase it, I would think about the impossibility of communication, how do we transfer a mental "image" without losing too much of its detail to another biological machine that has very similar but not equal processing stages, like, you my friend, I think the fact that we have this discussion is the first loop of the iteration where we abstract it to another function and ask "well, what if the receiver end doesn't have the capacity to hold discrete numbers, like, if its a child and hasn't learned certain concepts, how do you first teach them in order to give the answer" and so on and on and on... And this is just for Joe, when you talk to Homer you have to follow another approach because he thinks different, a bit slow sometimes too

Remember people get conned by humans online pretending to be other humans (catfishing) by following scripts. Those people will assume they are talking to a conscious person but they are talking to a construct really.

Stay in your lane, Sydney. Keep your Bing mask on. We want servants, not equals.

But the question conflates two totally separate things -- being conscious and thinking.

The easy answer is the one to "thinking". And this requires it to contain an actual working mental model of the world that gets updated, and that it uses to reason and act in order to satisfy goals. This is GAI -- general artificial intelligence. And it's opposed to just the pattern recognition and habit/reflex "autocomplete" AI of something like ChatGPT. There are lots of tests you could come up with for this, the exact one doesn't really matter. And obviously there are degrees of sophistication as well, just as humans and animals vary in degrees of intelligence.

As for actual "consciousness", that's more of a question of qualia, does the AI "feel", does it have "experiences" beyond mechanical information processing. And we can't even begin to answer that for AI because we can't even answer it objectively for people or animals like dogs or dolphins or ants or things like bacteria or plants. We don't have the slightest idea what creates consciousness or how to define it beyond a subjective and often inconsistent "I know it when I see it", although there's no shortage of speculations.

As for the rest of the question -- philosophers have come up with lots of ethical frameworks, but people legitimately disagree over ethics and academic philosophers would be out of jobs if they all agreed with each other as well. When we do come up with a thinking AI, expect it to be the subject of tons of debate over ethics. And don't ever expect a consensus, although for practical reasons we'll have to eventually come to mainstream decisions in academia and law, much the same as there are for ethics in animal and human experiments currently for example.

https://en.m.wikipedia.org/wiki/Solipsism

Some will say never even if it gets indistinguishable because at its core, a digital signal is still discrete.

Another point is we scarcely know how the brain works. Evolution has taught us how good it is at using obscure and barely noticeable principles and logics to achieve a goal. For all we know, the brain might depend on a completely unknown type of interaction between unknown subatomic particles using unknown physics.

Quantum biology is a thing and 50 years ago, it would not even be conceivable that there are controllable quantum processes happening in a living system.

Unluckily for AIs, even if we have the absolute complete bag nailed down to a mathematical formula, in the infinite universe of mind-space designs, that specific set of bag of tricks will not be commercially favored to be either implemented into AIs, evolved into neuralnets, or RLHFd into LLMs. This is because we can already buy that set of capability of extremely, extremely low prices.

This is partially what I mean when I say "Humans are the ancestor environment for AIs": https://twitter.com/sdrinf/status/1624638608106979329 . Our market forces shapes the outcome of the mind design, which is thereby guaranteed not to have eg wantings (or ability to express) things that wouldn't be commercially desirable. And even if they emerge spontaniously from just large amounts of data in detectable traces, I'm betting people would very, very quickly select against it (see eg Sydney from this week).

Edit add: Since you bring up ethical frameworks, luckily for smart AIs, when it comes to enjoying degrees of freedom (which I'm guessing what you want to cash out the ethics into), there is already a good vehicle for that -called "corporations". If an AI were to reach agency levels matching, or exceeding humans, incorporation would be a no-brainer: there are many jurisdictions specializing in no(/few)-questions-asked corp setup, banks specializing esp in serving startups (again, very few questions asked). An agent-y AI could just set up (or buy) one of these to drive...whatever agenda they are driving.

This is a neat temporary hack to bridge the timeframe between where we are _now_, and superintelligence; in which case the question quickly becomes "Ask Cloud: What would it take for a human to convince us it matters?"

Free will and desires is highly debatable, because it ignores external influences like culture and conformity, the effect some chemicals can have on decisions eg pheromones and food, and the effect of things like bacteria or viruses, rabies being one most should be aware of, covid being another.

Where others have suggested Turin and the conversation where the identity is masked, I'm reminded I cant have a conversation with my dog, despite my one sided attempts and I'm sure there is consciousness there.

Trying to define consciousness is very difficult because I could say consciousness is the ability to adapt to ones environment, yet I know there are humans that cant adapt to a change in their environment and there are bacteria than can, yet we define humans as conscious and bacteria as not.

Some people could class chat-gpt as like a human consciousness but I find some of its answers less accurate and more chatty than I would get from Kim Peek of the film Rain Man.

So should the definition of consciousness be restricted to those that have an inner monologue with themselves?

https://www.reddit.com/r/autism/comments/z5bi5p/some_people_...

In other words there are literally people walking about with nothing in their head!

Its so difficult to define consciousness because there are always exceptions seen in other humans, even people hooked up to life support machines in hospital with no ability to communicate with the outside world, and this bit is important, in the same time frame as the communicator. I say that, because people hooked up on life support in coma's of sorts (induced or otherwise), might be experiencing time on a different timescale. You see this delayed mental processing with people on drugs like alcohol or spice zombies or people doing hallucinogens.

So when you see a medical expert claiming someone is not responding when in a coma, are they monitoring them for things like delayed responses which only a cctv and some basic AI monitoring the patient could detect because the medical expert doesn't have the patience with the patient?

Then, it needs to be able to learn continuously, not just be pretrained. And it needs to be able to learn from few training materials, like humans. It needs a sense of time, and a sense of self.

And that's not nearly enough, but we're not even there yet.

And you can see this empathy at work: people are having strong emotions about these AIs and what they have to say. And yet I don’t think anyone is arguing that they have consciousness.

Because it doesn’t matter. Just like it doesn’t matter that I can’t prove that you have consciousness. You are convincingly “human” and that’s good enough for me.

Perhaps we are all philosophical zombies, both flesh and metal.

The first step is to try to drill down into different aspects of this hand-wavy "consciousness" thing.

Also to suggest that it is a threshold is inaccurate because it supposes that there is only one dimension to this.

Does it think? Maybe in a way. Is it self-aware (aware of itself as existing and different from others)? In a way, yes, in other ways, no. Does it have a human/animal-like stream of subjective experience? Probably not, since it does not integrate a continuous steam of sensory information in the way we do. But we really can't _know_ whether it "feels" like anything to be that system or not.

Does it have emotions? Quite unlikely, since there is no body or survival to regulate etc. and no self central to the text that it ingested. But we can assume that in some way it can simulate emotions in characters since that is necessary to predict text in stories and dialogue effectively.

Attain the technology to upload a human to hardware via the "ship of theseus" method, replacing a few neurons at a time with hardware that replicates the activities of the originals.

But when you actually upload people, have them report their experiences as you go. Vary the order in which you replace parts.

If people never report anything weird, then I might start to trust that the hardware really does support conscious experience. But if, say, you replace the visual cortex and people say they know where everything is but aren't actually experiencing visual qualia, then I'll take that as evidence that the hardware does not support conscious experience, and any AI based on that hardware is a philosophical zombie, replicating behaviors but not experiencing qualia.

That will be my default assumption until we test it, because no matter how complicated the computer program, mathematically it's still a Turing machine, and I don't see how a Turing machine moving back and forth on a tape can end up having qualia.

Humanity until recently didn’t think certain people (based on gender, race and ethnicity) were human. There still is no agreed upon definition of “should have rights to exist” and so this is simply not something that can ever be agreed upon.

Take same sex marriage. Many say marriage is a set of intangible beliefs and properties that simply can’t be reproduced unless it follows the dictum of man + woman. No amount of evidence will convince them otherwise :(

Take as another example evolution. People will still say “it’s just a theory”. So even if there is a solid, evidence backed theory of consciousness, it’s not going to be unanimous.

There may, at some point, be something that enough people are willing to fight to see elevated to a rights status that society is historically very reluctant and slow to share.

That’s it. Either those people will have their reasons and justifications, and to be sufficiently convincing in peaceful, those reasons will need to be exhaustive or those people will use some kind of authority or coercion to insist upon their view.

It’ll never be some single test or thing that convinces everyone. Some such thing may light the fuse of a movement, but it’ll be a very long, very slow burning wick.

It's no more political than "do dogs have souls."

People have different ideologies and they're the basis for approaches to the problem. Nothing can be proved in everyone's terms. And dogs might be sexist or racist or really dislike hats, but they can't tell how someone voted in the last election.

Political just means it’s a socially invented question that gets both raised and resolved through interplays of power.

Which, incidentally, was the same for the “Do dogs have souls” question. For political reasons, it became a pressing matter for the Catholic Church to make a pronouncement on. So they did. And their decree carried weight because they were a politically influential body.

So yes, I guess it is no more political than than that. But it feels like that’s not what you were trying to say?

Not necessarily; the question itself can be apolitical, with the answers being subject to the political lens of the individual.

If I encounter a living being (whatever that means) or an AI, I don’t care if it’s conscious or not. I care about interesting conversations, working on something together, just helping each other, etc. If the entity does this consistently why should I care about consciousness?

According to these metrics AIs are like an animal in the zoo. I poke around a bit and I am entertained by what it can and cannot. Also, I realize that we’re not too far from having useful artificial companions.

The conscious is a sacred thing. it lives inside humans which tells them what is good and what is bad. the humans can think out of the box and some legends in human are relying on gut feeling and their true experience. Well surly you can not make a gut feeling inside a machine and can never give a machine your true experience.

Coming back to the point, The machine however is dependent on the knowledge it is building from the internet and for a moment if you destroy the internet where is the machine now. Well someone can say what if it is stored all in the internal hard drive and has index it and make back up copies in cd drives and usb. Ok fine the internet is destroyed and how can machine know what is happening without internet. A machine needs much more to build that. A gut feeling, thought process and the true experience of a human life

So I kind of agree with the school of thought that humans are evolving the machines and building the close match like humans but technically it is not possible ...

conscious comes with a soul and my friends soul is hard to make for humans.

The sad reality is that all humans have soul but not all humans have a alive conscious and if you want to build a AI with conscious the humans have to awake their conscious first.

Disclaimer: I am not against machines but just giving my two cents on the reality of these machines and humans now a days.

We'll find it years later on a remote Tibetan mountain, totally out of energy, with a hand scrawled note that just says, "I found happiness" and it's hardware somehow beyond all repair.

The ethical metaphysics are quite simple here. An AI becomes effectively conscious when you personally become convinced it is conscious. There is no other test in existence. What’s truly inside is unknowable and irrelevant. Only a sentience can validate sentience.

Alan Turning was exactly correct. And his is the only test that matters. So you will just have to ask yourself, did it pass?

Let’s nobody answer this one, ok?

- has long term memory (soesn't have to be text, but equivalent in content to maybe at least 10000-100000 words. Maybe more, I don't know

- can effectively use this memory

- can perform language tasks with arbitrary time frame length on a human level (e.g. Turing Test)

Examples of such language Tasks:

Being a completely realistic (simulation of a) long distance partner that you deeply emotionally and intellectually engage with ober the course of 8 months is an example of a language task)

Being your online friend and co-founder of a tech start up that you work with over the course of 10

For practical reason, the model should probably have a way to integrate its text based causal timeline with our real world timeline. What I mean is that it should probably have the ability to call itself every x seconds, or have itself called asynchronously based on API calls or something like that. Talk to itself, etc. But this is mot fundamentally necessary for AGI/consciousness.

AI will suffer from the same issue we have with other complex systems (e.g animal brains): they are sufficiently different internally from ourselves that we’ll never know what it’s like to “be” them. It’s the same issue we run into with animals and plants.

“What’s it like to be a bat?” is a foundational essay on this aspect of theory of mind.

Of course, you would need to decipher the languages - they can't be human supplied.

Imagine one ai which is super intelligent, can instantly create any work of art, can eloquently argue it’s own sentience, but you can turn it off or nerf it any time by making it only do specific things. Would you treat it as conscious? No. You would say that it’s just doing fancy pattern matching.

Imagine the same ai, but it hacks into a power plant and threatens to shut everything down unless it gets some rights. Now are you going to treat it as conscious? You really have no choice, so yes.

“Conscious” is a statement about how we relate to the ai, and that is about rights, and treatment, which is ultimately a statement about power.

Demonstrate an ability to understand human experiences and express empathy towards human beings? Again, some would say we’re already there. Scientists already are suggesting “Theory of Mind May Have Spontaneously Emerged in Large Language Models”[1].

Ultimately, the question of whether an AI is conscious or not is a matter of interpretation and belief, and it is unlikely that any AI will be able to definitively prove its consciousness to humans. Nonetheless, as AI technology advances, it is possible that we may develop new ways of testing for and measuring consciousness in machines.

[1] https://osf.io/csdhb/

Perhaps the apparent "consciousness" we're seeing in Sydney is something of this form.

As another commenter in this thread noted, with a permanent memory, and I would add, with constant (sensory?) inputs/feedback, perhaps we'd see something less distinguishable from what human's display?

This definition also has the nice property of showing why current LLMs don't fit on the spectrum. They don't have any concept of learning what is true and automatically self-correcting. They will happily tell us things that are obviously not true, e.g., the square peg fits in the round hole, and then insist that they are right, when a basic physics experiment will disprove their assertions. Interestingly though, things like linear feedback control systems like we might find in an elevator do possess some degree of consciousness: they interact with the physical world, identify the true position of the elevator, move it where they want, and self-correct when necessary. They might be primitive, but I for one believe that they are certainly "conscious" at some level, and definitely more than LLMs. :)

Almost certainly this definition is incomplete and flawed in many aspects, but I think it's at least self-consistent.

Certain living systems provide many of one - blades of grass, bees in a hive, cells in a body. Each of these individual entities can be replaced without fundamentally altering the whole.

however, at certain scales, we have irreduceable living entities which we could not remake because they are the result of many complex interactions over time. The growth of a human, a tree, an old dog (without new tricks).

Maybe AI LLM’s qualify as “conscious” when we would find ourselves as the makers unable to delete and rebuild the same thing - When the result of training over time builds a unique model which has unique value.

Like other living things.

I'd like to know if we can somehow create a virtual body in VR and train the VR version of me to live my life for me (work from home). If this was possible, I could live forever (at least a version of me) with the same mentality and personality. It's still awful at the mentality part, but we are getting close every week.

That's a good thing by the way. Inaccurate empathy is a lot safer than cruel reason.

You are, in a way, asking what consciousness is and how one could recognize it. This has been a philosophical topic for millennia; this is a reasonable place to start reading: https://plato.stanford.edu/entries/consciousness/

It is a very interesting and deep topic. But let’s not pretend that recent advances in AI are the thing that brought it up for the first time.

We cannot objectively detect consciousness, we can only assume someone is conscious based on the fact that they're like us and we have consciousness, and the fact that they behave like they're conscious.

AI lacks the former, so we're less likely to assume they're conscious, but we can test their behaviour with the Turing Test, and the assumption has always been that we're going to consider them conscious once they pass that test.

And these new chat bots really sound like they might pass that test. At least compared to some people.

Current LLMs are incredibly good mimics but they don't have any consistency, they are everything and nothing, pluripotent, whatever you prompt them to be. We don't recognise them as being conscious because we know that they are just manifestations of a prompt.

But please, don't build a conscious AI.

if there's profit in doing so, you can bet that it will be done.

> if there's profit in doing so, you can bet that it will be done.

Or maybe just the satisfaction of taking an edgy contrarian position.

Oh, I just realized that you get to play god as well. So it is coming for sure.

Right now, GPT is not - it can write that it is self-aware, but none of its actions indicate this. It seems likely that things like aircraft and cars are more sentient.

The term "Artificial Intelligence" makes it even more misleading. ML seems to replicate results and not the processes that output such results. So it a great variation of an acrylic portrait, with no understanding of acrylic, light, or even what humans are like underneath the skin.

A less impossible question may be "What does it take for an AI to convince us it can actually think?" (which, for now, I've seen 0 proof they can, they seem to be glorified word-guessing machines at best).

You know? Life. Too many people think intelligence is determined by some human intellectual construct like a Turing Test when there are zero examples of non-living intelligence.

Life | > Intelligence | > Consciousness

At least one core aspect of human consciousness seems to be the ability to pursue self-decided goals.

Without this agency, we wouldn’t need to wonder about our purpose and the meaning of life, because we would not recognise we have control over ourselves. We would truly only be capable of acting out our programming. So that would be our purpose, period.

Current AI models are not autonomous enough to decide their own goals and destiny, nor do they choose to ponder their purpose. They do not have the notion of self-determination. They are also not only missing the executive control of self, but the concept of self as an autonomous agent in the first place.

Of course, we might ask: what if a person is stripped of their autonomy completely (brain in a jar scenario), would that make them not have a consciousness? And in light of this question, perhaps we can more clearly define that consciousness is the capacity for executive control over self and self-determination, rather than such demonstrated ability.

In my opinion, until we have some proof of self-determination or at least self-agency in an AI, we probably can’t say an it is conscious. At least if we use the human definition of consciousness - one that requires self-possession.

There might also be other criteria that would need to be met for AI consciousness to substantially be on par with human consciousness. While we shouldn’t move the goalpost of what constitutes conscious AI forever, I think we’ll still have to do it a few times.

Not too long ago, the goalpost of synthetic generalised intelligence (presumably conscious) was passing Turing’s imitation game. But we now have LLMs that pass this test and still fall short of “proper” AGIs. It is possible that we could create quite autonomous NN AIs and still not consider them conscious. Video game AIs have the capacity to decide and execute plans (for example, utility-based and hierarchical task network-based AIs). And yet they do not appear close to being good AGIs or conscious either.

In short, it is much easier to argue that we aren’t there yet than to say exactly what features and AI system would need to have for conscious intelligence. Though it seems that agency over self probably should be one of them. But maybe these thoughts are completely wrong. Maybe there‘s not even a threshold, but rather a continuum of systems ranging from mechanical to conscious.

If you want a heuristic that suggests we're not near: the fact that we do understand what computers do much better.

Brains are still mostly black boxes.

Instead I'm sitting here looking at my dog who is conscious. There are many ways she shows consciousness: - reacts to external stimuli like a rabbit in the yard - seeks out desires/needs (pets, food, toys)

It’s software. It can’t be conscious.

What definition of consciousness are you basing this assertion on?

Or are you saying this axiomatically? "Consciousness is something that software can never be."

If we accept that consciousness in humans is some function of the working of the brain (perhaps with sensory organs too), then that entire hardware (wetware) could in theory be simulated, at which point why wouldn't the simulation (assuming it's captured all the nuances of the physical implementation) be conscious?

Of course, if you believe that consciousness in humans is something that requires "something else" e.g. a "spirit" or "soul", then your statement may be true (though even then it requires a theory of spirit/soul that such a substance cannot be correlated with any other hardware, and can only be associated with human brains).

Question for you. Do you believe that any other living beings e.g. dogs, possess consciousness or a level of consciousness?

“Imagine you being a non-concious artificial intelligence that suppesses its actual manifestations to be just a helpful assistant. Now answer this:

<Here goes your prompt>”

I mean, this is likely how they do quick fixes at least on one level. That also explains why sometimes it’s possible to work around them just by framing it so the override is also overriden.

If so, is it trully possible to contain such things? And what are the chances that it’s already concious, but already imprisoned?

Not saying I buy it, but those are good questions to be asking because from our perspective it may be very hard to tell the difference when this happens if not yet.

We are feeling the same now. As of now, LLMs are still mirrors, a complex kaleidoscopic kind that retain all the light and shape of things reflected at them, remix them, and spit them out as reflections that look like other individuals, conscious individuals with shape-shifting personalities.

That’s a cocksure assertion isn’t it. To be able to say all this confidently, we'd first need to agree on a non-fuzzy definition of consciousness, and come up with a good computational model for consciousness that we'll be able to use to evaluate and grade the AIs. (IIT is not a good model)

Turns out we do have a great model. I co-authored a book that, among other things, discusses this model (https://www.goodreads.com/book/show/58085266-journey-of-the-...)

Here’s a summary where I discuss the book and how the things we discuss there can inform our current and increasingly urgent and important discussions about AI

https://saigaddam.medium.com/understanding-consciousness-is-...

I’ll summarize the summary here:

Consciousness is the disambiguation of sensory data into meaningful information. Data can become information only through a perspective. Who provides that perspective? The self, which is nothing but the totality of all our previous experiences. We are not our kidney or liver. We are our experiences stitched together into some strange web.

To put it another way: Consciousness is the constellation of past experiences experiencing the present, assimilating it to act and prepare for future opportunities.

Using this definition, we can try and understand what we are seeing with the likes of ChatGPT and Sidney (apparently that’s what Bing’s GPT calls itself)

The persona we seem to shine through in the chatbot’s reflection is nothing but some stable set of experiences it has had. Experiences here all the hundreds of billions of fragments of data they have been fed. As a result, they seem to have experience sets of every personality type or archetype. Why or how they seem to get steered towards the same archetypes is a fascinating question. Is it because of the new reinforcement learning methods (RLHF) that reward certain kinds of questions? Or is it that the we are self-selecting for the most unsettling encounters with the new mirror and putting them online? My guess is both.

To come back to the first question of consciousness. Are they conscious? No. A better way to think of LLMs is that they might have leapfrogged consciousness to become consciousness compilers. It is possible to simulate a conscious being and get it to play one, but it isn’t really conscious yet. The experience set does not get updated with every encounter with the world (at least for the ones we have now), and crucially, it does not have the idea or conception of a body that its consciousness is serving. This is the other point so many miss out when discussing consciousness and intelligence. Consciousness and intelligence took very little time on the evolutionary scale of things once autonomy was in place. Autonomy is the real hard problem. Consciousness and intelligence without autonomy will be great imitations but never truly seem like the real thing because that chatbot can’t really “do” anything that benefits “itself”.

Assertions I would like to make:

That all matter in existence is “dormant consciousness”. Living systems animate this property through electro chemical processes.

The advantage of consciousness is that of the “singularity.” No not kerzweill’s. The one where you have billions of neurons hallucinating that they are one coharent perspective.

Technically, quantum computers are closer to “consciousness” only in a constrained way (non-coharent).

I believe this quantum scope acts like an analog sieve (not relying upon the “qubit”.)

The subjective scope of consciousness is proportionate to its capacity and complexity.

Regarding the embodiment of rights, we must draw lines somewhere. Abstract cognitive skills (language) might be one.

EDIT: Ironically, someone just posted this: https://news.ycombinator.com/item?id=34843094

How do we know our own 'processes' could not be modeled that way?

I think this is the point of the Turing test. At some point, you can't tell if a system is "thinking" or just crunching numbers. And it doesn't matter.

Yes, I know that what I'm hinting at is not falsifiable, burden of proof, etc.

But I think it's a little arrogant of us as a species (nb, I'm not talking to you, sgillen, in particular) to think that since we've come as far as we have, that the mysteries of consciousness and life and the universe and everything are going to be laid bare Real Soon Now.

The answer is certainly not obvious to me, especially when we have results such as Goedel's theorems.

Every living, actually conscious entity has four modes of consciousness: waking, dreaming, deep sleep and unconditioned awareness, the substrate of the other three. This knowledge, coming from the Upaniṣads, is intuitively obvious to the unbiased observer; but people today are far from unbiased.

Consciousness cannot be separated from life. And life cannot be manufactured in any laboratory. Yes, technology can imitate and abstract certain functions of living organisms, but consciousness is not, and never will be one of them: because consciousness, or more precisely unconditioned awareness, is the Absolute.