"Consciousness" requires definition. As a neurologist, I use the term in as many as four different ways depending on the circumstances. Let's see if we can box it in for the purposes of this discussion by defining some related concepts:
Reflex: A simple deterministic system. Input -> output. Neurons are connected in series.
Cognition: implies a complex system in which concepts govern the relationship between input and output. Input -> concept -> output. A concept is a set of linked attributes, in the sense that "hairy," "barking," "four-legged" and "smelly" might link to form the concept of "dog." Linkage between attributes represents an implicit theory. Theory = meaning. (See Bernstein, "A Basic Theory of Neuropsychoanalysis.")
Neural network: a computing system made up numerous simple processing units that are massively interconnected. They are hooked up in parallel, as well as sequence. Cognition is the characteristic behavior of a neural network. All brains capable of thinking are neural networks. Artificial neural networks can be thought of as models of the brain, albeit with important qualitative and quantitative differences. Intelligence: a parameter of cognition. Anatomically speaking, intelligence correlates with the degree of connectivity of neurons in the brain. Humans have more connections than animals. Einstein had more connections than me. When we say something is intelligent, we mean that it has the capacity to carry a lot of information in the form of concepts and theories. Concepts can be numerous, subtle, and hierarchical. Theories can be deep and valid.
Sentience: is a superset of intelligence that implies the presence of consciousness.
Consciousness: remains to be defined. We can call it "self-awareness" for now.
The "deterministic" system described by OP is neither intelligent, nor conscious. The computer system OP describes is reflexive.
Not all computers are reflexive. Suppose the task were handled by a self-organizing neural network. The formation of concepts that drive output implies theory formation. Even though these theories are primitive and idiosyncratic, we could say that such a system has rudimentary intelligence.
To make this computer seem human, we need to build in enough processors, and a massive enough degree of connectivity to approximate the capabilities of the human brain, both in terms of raw storage capacity, and also in terms of the capacity to form hierarchical concepts. We would also need to build in emotion. "Emotion" may require assimilation of as few as three attributes: valence (punishment vs reward), intensity, and whether an approach or withdrawal response is called for.
I don't know that it's possible for us to design such a system. But suppose we could. Would it be conscious?
Perhaps not, because of the question of "self-awareness."
I dislike the term "self-awareness" because it's too easily defined in ways that are trivial or circular. It means something (perhaps not much) if you are aware of my self. Anything with a brain, and any computer, can do that. As to whether you are aware of your own self -- of course you are. You ARE your own self.
The concept of "self-awareness" only has meaning, then, if the nature of the self is ambiguous.
The question then becomes, under what circumstances would a computer be faced with an ambiguous self-nature?
To date, we have not evidently seen the need to design such a computer. Ambiguity is a bug, not a feature. To be clear, the more complex the system, the more difficult will be the design task, and the more unpredictable the result. But it will be a system nevertheless, one that will render its own particular output perfectly. Such a system will have no basis to contemplate or even to define its "self" in any meaningful way.
But what if we were to design a computer that could evolve? If it could perceive that it were changing over time, would that be enough ambiguity to cause it to consider the nature of its self?
I doubt it. All humans evolve, in a sense. We mature over time. But the change is seldom dramatic enough to cause us to question our own nature.
With regard to the question of the Self, humans are struggling with bigger issues. Priests tell us we are body and soul. Philosophers tell us we are an existence and an essence. Psychoanalysts tell us we are an Ego and a Self. Are these conundrums unique to humans? Or can computers get in on the action?
Possibly. The question is, what is the nature of intelligence? Where does it come from?
So far, we have been talking about intelligence as a form of information that arises as an emergent property of a complex system, in the sense that theories and concepts represent the work product of a neural network.
We also hinted at the obverse; in other words, the notion that complex things are based on information. When we talk about designing computer systems, for example, the "design" part refers to information. If a computer design is based on a blueprint; we can look at the computer as a physical manifestation of the information space outlined in the blueprint.
Likewise, our solar system is a physical manifestation of information. Information in the form of gravity came over from the infinite beyond into the "3+1" material world at a time when the universe consisted of rapidly expanding gases. Gravity caused gas clouds to coalesce into stars and planets. The structure of the information space carried in gravity implied the structure of the physical universe, including our home.
The human brain is a manifestation of an information space, and in spite of what one might have heard, we are nowhere near understanding the nature of that information. We have decoded the human genome, and found it to account for little of the brain's structure. The most important gene that distinguishes the human brain from that of animals is ARHGAP11B, which allows for wild chaotic branching of neural connections. Hebb's principle -- "neurons that fire together, wire together" -- then accounts for the development of the neural network. To say this is a non-deterministic process would be an understatement. In addition, there is evidence that the brain has fractal structure (www. stat.wisc.edu/~mchung/ teaching/MIA/reading/ fractal.kiselev.NI.2003.pdf - remove the spaces) and so there's another information space that needs to be accounted for.
What is the information space that defines the brain? And what (if anything) does that have to do with our concept of "consciousness?"
If consciousness is only ever a property of a distinctly human intelligence, I would submit that we cannot possibly design a computer that would have a consciousness separate from that of its maker. No matter how human it seemed, it would always be the puer eternis, never able to differentiate its Ego from the parent. (Might still try to kill us, but that's another story.)
But. To the extent that anything we imply in the word "consciousness" precedes human intelligence, we might be in trouble. Note well, the laws of thermodynamics suggest that it's a downhill run from star nurseries to intelligent life. If so, the information that gave rise via gravity to primordial star nurseries preceded and implied human intelligence. In that case, we can expect intelligence to stay in its meat suit as long as that's the best way to fend off the Second Law. When it encounters a more efficient solution, it will jump ship.
In summary, the answer to your question is:
1. The brain does not work like the computer you describe.
2. But, it is theoretically possible to design a computer that does. Whether or not such a thing is a practical possibility is another matter.
3. We can't know if such a computer could become sentient. Under some circumstances, it would be inevitable.
