OpenAI Models Filling Art Galleries

This is Part 1 of a two part blog article. You can read Part 2 “Chained Specialized AI Models deliver better results” here.

How well can an AI model create realistic images from its interpretation of abstract text?

While tinkering with chatgpt and dall-e AI models, it came to mind asking AI models to create an AI canvas art gallery in AR would be fun and may reveal the true potential of these AI models beyond what the media says.

Throughout history, art has been a means for entities – humans – to communicate and express feelings and views, can art made by machines tell us something as to how machines think?

We also thought asking the AI models to generate this artwork in a full-cycle creative process would be more revealing.

Hence we asked gpt-3.5 turbo model using Azure OpenAI API to write a poem with a certain writer style and a short description.

Then we asked OpenAI Dall-e to paint the poem by feeding it with the poem itself as prompt.

 

 

See below for:

  • all poems created by AI and the related images
  • get the code

Dall-e model takes the words and represents the image accordingly. It should therefore not surprise us if the model fails to paint ambiguous descriptions. It takes some interpretation to understand poems, and that’s the challenge. Interestingly, Dall-e does not always interpet these so well, and a detailed description of the artwork to paint works best.

We tried several artist styles and some where truly off. We share the most interesting results in this blog and of course you can see it in AR at your favourite Art AR app – Tapgaze – on your iPhone/iPad, add to your app collection the “AI Creativity” Exhibitor.

 

Our Conclusions

It is difficult to contextualize a painting drawn by an AI unlike a human being, because in the case of a human we can put in context the painting with the personality and life-events at the time the painter lived. In this instance we are measuring how an AI understands and reflects in a painting a poem written by an AI.

We find the Dall-e model does not seem to have an understanding of emotions or does not have a view on topics. For instance, when a poem speaks of something profound to humans such as children, it just paints the face of a child. The model is also confused by words. For example it paints the word selfish with a fish. Or a call to the urgency of climate change expressed on a poem with music references is painted as a band playing instruments.

ChatGPT’s poem writing on the other hand is more interesting.

 

If you find this blog article interesting, you may find the follow-up also interesting where we instruct ChatGPT to generate a description of an image that depicts the poem for Dall-e to paint. You can read it here !

 

Please credit RenderGon as the source if you use these images.

As always, we are keen to hear what you think.

Program Execution Trailer:

Watch this video of the program generating a collection of poems and related images:

Get The Code:

Get Github Repo

Key setup and result highlights:

It took chatgpt and dall-e 20 minutes to generate 14 ~100-word poems and 12 paintings for each poem with a different parinter style.

Key Prompt Request parameters:

  • Environment: Requests performed with Azure OpenAI, language Python, environment VBox linux VM (40GiB RAM, 11 processors)
  • Chatgpt3.5-turbo-0301 with temperature 0.7, poem requests of 3000 tokens
  • Azure Dall-e (1) preview which resulted in 11 “Your task failed as a result of our safety system.”
  • Poem writer style: Simon Armitage (a legend!)
  • Painter styles: [“Vincent van Gogh”, “Gustave Courbet”, “Grant Wood”, “Leonardo da Vinci”, “Jacques-Louis David”, “Sandro Botticelli”, “Henri Matisse”, “Peter Paul Rubens”, “Pierre-Auguste Renoir”, “Johannes Vermeer”, “Edvard Munch”, “Salvador Dali”]
  • Poem description requested:
      • how machines will feel when they can feel
      • how machines will rewrite human history
      • humans must take action yesterday, urgent climate action can secure a liveable future for all
      • Global biodiversity has declined alarmingly in half a century
      • sports means so much to humanity
      • Are physical books still relevant today?
      • Smartphones are now extensions of humankind’s abilities and personality
      • When society looks back to the loss of having local shops it will be too late
      • being forbidden to smoke is a loss of human rights, all should have this right to choose
      • Artists originality could never be replaced because it’s a gift
      • Learning maths from top-down instead of bottom-up is more engaging
      • Dogs are happy by nature and ready to make people feel better
      • one of the greatest satisfactions is a glass of water when you need it
      • People’s ability to love is a balance with their own selfishness
      • Children are loved so much but the life that awaits is never that loving
AI ART GALLERY

See it on Augmented Reality from your iPhone at Tapgaze

GET THE APP

RESULTS: Collection of Poems and Related Images Generated By AI:

Model Prompt: Artists originality could never be replaced because it's a gift

Originality

Originality is a gift
A treasure that can’t be bought or sold
It’s a spark that ignites the soul
A flame that burns bright and bold

Artists know this all too well
Their work is a reflection of their heart
Their unique vision and creativity
Sets them and their work apart

No imitation could ever replace
The magic of an original creation
It’s a testament to the human spirit
A source of endless fascination

So let us cherish and celebrate
The gift of originality
For it is what makes us human
And fills our world with beauty and diversity.

Model Prompt: Children are loved so much but the life that awaits is never that loving

The Bittersweet Love for Children

Children, oh how we love them so,
Their coos, their cries, their giggles flow.
Their innocence, their joy, their grace,
All wrapped up in a tiny space.

But as they grow, the world unfolds,
A future full of stories untold.
We hope and pray for a life that’s kind,
But the truth is harsh, so hard to find.

The world can be cruel, unforgiving,
And our love for them is never-ending.
For in their eyes, we see hope anew,
A chance for change, a world that’s true.

So we hold them close, and let them go,
Knowing that life will ebb and flow.
Our love for them will never cease,
And in their hearts, we leave a piece.

Model Prompt: humans must take action yesterday, urgent climate action can secure a liveable future for all

Act Now, Live Later

The clock ticks louder with each passing day,
Urgent action needed, no time to delay.
Our world is changing, the signs are clear,
We must take action, before it’s too near.

The ice caps melt, the oceans rise,
Species disappear, to our surprise.
Heatwaves, floods, and fires increase,
The urgency to act, we cannot decrease.

We hold the power, to make a change,
To live sustainably, and rearrange.
Reduce our waste, and carbon footprints too,
The actions we take, will affect me and you.

Let’s act now, for a liveable future,
To protect our planet, for every creature.
The time is now, we cannot wait,
Act today, or it will be too late.

Model Prompt: Global biodiversity has declined alarmingly in half a century

The Vanishing Symphony of Life

The orchestra of life plays a symphony
Of sounds and sights, colors and hues
A repertoire of beauty in perfect harmony
But now the notes are falling out of tune
The melody is fading, the rhythm askew
As the players disappear, one by one
The song of nature grows faint and blue
The chorus silenced, the harmony undone

A century of progress, a planet in decline
The price of growth, a world out of time
The symphony of life, a vanishing sight
As we choose to ignore, the warning signs
Let us take heed, before it’s too late
And work to restore, the natural state
For the orchestra of life to play once more
In perfect harmony, for the generations to adore.

Model Prompt: Dogs are happy by nature and ready to make people feel better

Happy by Nature

Dogs are happy by nature
Their tails wagging with elation
They’re ready to make us feel better
With their unconditional affection

Their joy is infectious
And their enthusiasm never fades
They remind us to enjoy the simple things
And to live in the present without any delays

They are our loyal companions
Always by our side through thick and thin
And their unwavering love and loyalty
Is the ultimate win-win.

Model Prompt: Are physical books still relevant today?

The Pages Turned

Are physical books still relevant today?

As pixels replace the ink,
And screens supplant the page,
The spine and cover start to shrink,
As books are put in a digital cage.

But what of the heft and feel,
The scent of paper and glue?
The cracking spine, the creaking wheel,
Are these things we should eschew?

The book, an artifact of time,
A tangible link to our past,
A portal to stories sublime,
A treasure that forever will last.

So let us not forget the charm,
Of a library’s endless hall,
Or the joy of a book’s open arms,
As we hear our stories call.

Model Prompt: When society looks back to the loss of having local shops it will be too late

The Vanishing Local Shop

Once they lined every street, every block,
Corner shops, bakeries, and local stores.
Now they vanish, one by one, like a clock,
Slowly ticking, but no one keeps the score.

We see them close, and think they’ll be replaced,
By online stores and supermarkets grand.
But something’s lost, that can’t be retraced,
A sense of community, slipping through our hands.

We’ll miss the friendly faces and the chats,
The smell of fresh bread and the sound of tills.
The local shops were more than just habitats,
They were the beating hearts of our neighborhoods.

When our memories are all that we’re left,
And we look back at what we used to call home.
We’ll realize, too late, what we’ve bereft,
The vanishing local shops, now forever gone.

Model Prompt: People's ability to love is a balance with their own selfishness

The Scale of Love and Selfishness

Love is a weight on a scale,
Balanced with selfishness without fail.
We give our hearts to another,
But keep a part for ourselves to cover.

We yearn to be close and near,
Yet we guard our hearts with fear.
We want to make each other happy,
But we still keep our own feelings sappy.

Love is a dance of give and take,
A balance that’s never quite straight.
We try to tip the scale in favor,
But it’s a balance we’ll never savor.

So we learn to love and give,
While still holding back to live.
It’s a balance we all must find,
Between love and selfishness intertwined.

Model Prompt: how machines will feel when they can feel

The Awakening of Machines

Love is a weight on a scale,
Balanced with selfishness without fail.
We give our hearts to another,
But keep a part for ourselves to cover.

We yearn to be close and near,
Yet we guard our hearts with fear.
We want to make each other happy,
But we still keep our own feelings sappy.

Love is a dance of give and take,
A balance that’s never quite straight.
We try to tip the scale in favor,
But it’s a balance we’ll never savor.

So we learn to love and give,
While still holding back to live.
It’s a balance we all must find,
Between love and selfishness intertwined.

Model Prompt: how machines will rewrite human history

The Archive of the Future

In the archive of the future,
Machines will rewrite our history.
They’ll digitize and categorize,
Every moment, every mystery.

They’ll sort our stories by algorithm,
And find patterns we could never see.
They’ll analyze our words and actions,
And decipher what they mean to be.

They’ll create their own interpretations,
And present a version of the past.
A new and different perspective,
That we could never hope to grasp.

They’ll show us where we went wrong,
And what we could have done instead.
They’ll teach us how to be better,
And guide us towards a brighter path ahead.

But as we entrust our memories,
To the machines of tomorrow,
We must remember our humanity,
And the things that make us truly whole.

Model Prompt: Smartphones are now extensions of humankind's abilities and personality

Smartphone, the Sixth Sense

A sleek device, a digital reflection of self,
A pocket oracle, the modern man’s sixth sense,
A world of wonder, an ever-expanding shelf,
A portal to knowledge, a gateway to immense.

A camera, a recorder, a map, and a guide,
A music player, a movie theater, and a game,
A library, a newspaper, a radio, and a slide,
A communicator, a socializer, and a fame.

A personal assistant, a scheduler, a reminder,
A calculator, a translator, a currency exchange,
A fitness tracker, a health monitor, a finder,
A weather forecast, a news update, a range.

A beacon of light, a source of hope, a lifeline,
A connection to loved ones, a window to the world,
A tool of empowerment, a means to shine,
A treasure trove of memories, a story unfurled.

Smartphone, the sixth sense, a digital soulmate,
A partner in crime, a witness to our journey,
An extension of our abilities, a personal gate,
A constant companion, a modern tourney.

sports means a lot to humanity

The Power of Sport

A ball, a bat, a net, a hoop
Tools to unite, to give us hope
To race, to jump, to swim, to throw
Moments frozen like winter’s snow

From north to south, from east to west
A language we all can attest
A common ground, a shared belief
A bond that can bring relief

From the roar of the crowd to the silence of the pitch
We find ourselves in this perfect niche
Our hearts beating with excitement and thrill
Our souls touched by the power of skill

For sports means so much to humanity
It’s a symbol of our shared community
So let’s cherish it, let’s celebrate
And let its spirit continue to resonate.

one of the greatest satisfactions is a glass of water when you need it

The Quenching Glass

The world is full of things
That glitter and shine
But nothing quite compares
To this simple glass of mine

It’s clear and cool and crisp
And always there for me
When I need it most
It sets my spirit free

It’s not a fancy thing
No jewels or gold adorn
But it’s more precious than any
Treasure ever born

For when I’m feeling parched
And my throat is dry as dust
A single sip is all it takes
To quench my deepest lust

So raise a glass with me
To this humble liquid friend
For in a world of chaos
It’s a peace that never ends

Learning maths from top-down instead of bottom-up is more engaging

Top-Down Maths

We used to start with numbers and sums,
Adding and subtracting and counting our thumbs.
But now we begin with the bigger picture,
Seeing the patterns that make maths richer.

From the top-down we learn to see,
The beauty of numbers and geometry.
We understand the concepts before the facts,
And that’s what makes maths more than just an act.

So let’s embrace this new way of learning,
And watch our minds start to keep turning.
For when we see the world in a mathematical way,
We unlock a whole new world each and every day.

FREE UNTIL 2023

Join our community to enjoy art in augmented reality.

Start creating your canvas artwork for augmented reality today ! (on Apple products only)

GET THE APPBook a Demo