The Mind’s Image Processing

Lossy data compression methods have captured my attention and imagination from orthogonal sources (day job and Victus Media)

A Curious Coincidence: Photons, Neural Compression, and Perception

Long have I wondered how our minds perform the enormous processing task of transforming incident light into a 3 dimensional reality, while triggering memory and understanding. After a great deal of reading and many years working on sensor simulations I have only a laymen’s understanding of the process. Join me in a journey that crosses thousands of light years to illuminate the magic of our minds.

Photons from Distant Galaxies Become Information to Our Minds

Let’s begin by tracing photons from a distant quasar* at the edge of the universe all the way to a human observer looking at telescope data. The photons first leave their energetic home and traverse space and time over billions of years. Their course may bend many times as the photons near other galaxies and eventually pass close to our own sun and encounter an enormous aperture. Let’s gloss over the complex signal processing that transforms data and imagery, and instead follow the newly made light from a human readable display. The light finds it’s way to curious human eyes. For the next step in the information chain, I’ll borrow from an earlier post on digital and optical Images:

Our optical system quantizes measured photons in a similar manner to a digital focal plane. While rods give us black and white vision in low light conditions, our cones spatial orientation uniquely captures incident photons. While the majority are sensitive to red light (64%), some to green (32%), and only a tiny minority to blue light (2%). The proteins in the cones help trigger a nerve response which is channeled down the optical nerves to the part of brains that transforms these signals into three dimensional representations of our surroundings.

Local light that comes from sources closer to home, like this blog post or your display, are raw data that your senses use to compose a conscious image of your surroundings. But how do these incident photons become a human concept? One vital piece of the puzzle is massive data compression.

Biological Optical Data Compression

The investigators calculate that the human retina can transmit data at roughly 10 million bits per second.. That data rate matches the older Ethernet cable speed of 10mbits/second.
What data rate is available at the visual cortex?
article in scientific american (behind a paywall)

Visual information, for instance, degrades significantly as it passes from the eye to the visual cortex.

Of the virtually unlimited information available in the world around us, the equivalent of 10 billion bits per second arrives on the retina at the back of the eye. Because the optic nerve attached to the retina has only a million output connections, just six million bits per second can leave the retina, and only 10,000 bits per second make it to the visual cortex.
After further processing, visual information feeds into the brain regions responsible for forming our conscious perception.

Surprisingly, the amount of information constituting that conscious perception is less than 100 bits per second. Such a thin stream of data probably could not produce a perception if that were all the brain took into account; the intrinsic activity must playa role.

Yet another indication of the brain’s intrinsic processing power comes from counting the number of synapses, the contact points between neurons. In the visual cortex, the number of synapses devoted to incoming visual information is less than 10 percent of those present. Thus, the vast majority must represent internal connections among neurons in that brain region.

So our optical process distills billions of potential bits of light into 100bits per second. Although the compression is lossy, it’s good enough for us to interact with our surroundings and respond to motion rapidly (reflexes & frontal cortex). My hypothesis is that our mind continually estimates our surroundings and updates that state with a miniscule fraction of the raw input data (the ultimate change detection). I can only guess that massively parallel pattern matching is taking place to compress vital data differences at each moment. This compressed collection of neural signals updates our perception of reality. We continually track and estimate the world around us.

*Quasar: there is now a scientific consensus that a quasar is a compact region in the centre of a massive galaxy surrounding the central supermassive black hole. (from wikipedia link at the start of the post)

^Visualization of data: Purely visual representation of distant solar bodies has been a challenge to artists and masters of scale (exaggerating structures for display), and this is how the non-experts experience the data

Be Sociable, Share!

  • Laurent Boncenne

    I like the thinking and the possible idea that it could be possible to apply such pattern into software/web.

  • Mark Essel

    Wow Laurent, you're batch processing posts today. Glad to see you enjoying a few of my thought riffs and finds.

    I'm still amazed at how little information our minds need to update knowledge of our surroundings. If only web updates could be so efficient to our state of knowledge.

  • Laurent Boncenne

    The fact that we forget might be where it's at. Forgetting and succinctly storing what we learn/do/share/make gives us in my opinion the ability to get more of these updates.

    Web updates come in roughly and there is a double process of selection to make.

    #on thought riffs and finds :
    It seems we have both relevant content to share to each other, some that I can't unfortunately share with my close friend because of their interests. (always tried to get my friends to visit stuckincustoms lol :-D, you do, saves me time)
    Then again, having irrelevant content is also a good way to keep in touch and actually be aware of those surroundings…

  • Mark Essel

    I'm a huge fan of Trey Ratcliff (admit to not picking up the book yet, will remedy that asap). I fire a storm of links to his photo-blog whenever I can. Not only his perspective but his way of uplifting other photographers and going on photowalks has earned my trust/admiration.

    I keep joking that my fiancé Michelle and I would love to be his pack handlers on one of his excursions…

  • Pingback: EPIC, the end of email

  • Ellie K

    This isn't a doozy of a post, your words. It is an easy read and accessible without getting highly technical in any single area. A few thoughts an questions I now have, hope you'll reply despite 4 mths since post date!

    There are 4 interfaces. each with significant “packet dropping”:
    1. Light enters the eye at lens and pupil, passes through vitreous humor, travels diameter of eye and transitions to
    2) Retina which processes then sends input to
    3) optical nerve to
    4) visual cortex to
    5) specific physical area in brain where visual signals are merged with other senses to give composite perception of world around us.

    I was aware of the power of our retinas, but didn't realize how little information passes on to the visual cortex and brain. Yet it must be sufficient, else human image perception and synthesis would not be what it is. So how do we manage, with such “lossy” data feeds? You suggest massively parallel pattern matching, running dynamically. Sounds possible. And if only 10% of visual cortex synapses handles incoming data, then the other 90% is processing and evaluating and synthesizing dynamic input AND storing as well. So here's my question, is it possible that the already established synapse connections work like cache or “temporary files” further speeding up visual perception?
    (Sorry for long post)

  • Mark Essel

    Don't be sorry for the comment, many folks have left far longer and I always appreciate well thought ideas.

    Yes, I think our visual processing center is quite capable of caching non-changing information in the short term but I don't have any further information beyond that (my neuro-biology background is light, only a single introductory college course).

  • Dentist Crawley

    good article .. .well done. I will decidedly subscribe for this blog

  • Mark Essel

    Glad you enjoyed it, you may prefer the “far out” topics to some of the startup/coding based ones. You can find topics written about at the tag cloud page.

  • Pingback: What Karate Kid and the Devil’s Advocate have in Common » Victus Spiritus

  • Pingback: Coding web apps is the new literacy » Victus Spiritus