December 9, 2017
In the next few years, I am going to see software alone reproduce and automate, to a standard good enough for most, the finished edits that I now achieve manually. This won’t take away the pleasure of editing individually, but once this occurs, the real skill needed will be in guiding the software. Providing the human insight. Who is guiding the algorithm developers? What values are they promoting? AI will learn, so we need to teach it what matters most. Who gets to decide who has the most input in this process? What values will they embed into their technologies?
1
This feels like a good place to begin.
The above note was written at the end of 2017, quickly typed out on my phone after I had just seen some examples of machine learning powered photo-editing capabilities coming out of Nvidia Research. They were difficult edits to achieve convincingly using the current (manual) post-processing applications and techniques of the time: seamlessly changing the scene from the middle of a sunny day, with its hard edged shadows and high contrast lighting, to deep night, with its convincing soft diffused shadows across a series of photographs - including images of high-rise apartment buildings, with their multiple rows and columns of shade casting awnings and reflective windows. And I was, to put it mildly, equal parts impressed and terrified with the results.
In 2017, anyone who had spent time experimenting with methods to achieve this knew how challenging it was to render this kind of adjustment seamlessly; not always impossible, depending on the source image, but the sort of task that required an immense amount of time, precision and care. It called upon both a sophisticated level of visual acuity, aesthetic sensibility, value sensitivity, technical experience and good old fashioned mechanical dexterity. Most of the time, it wasn’t really worth even attempting. “Can we change day to night?” was the kind of question I’d probably just respond to with “it would be easier to just re-shoot it at night.”
And so there I was, looking at low resolution, but technically and aesthetically successful edits, utilising machine learning. At the time, I was almost entirely unfamiliar with the field of ML, outside of some early attempts at style transfer that had my curiosity piqued. I had spent some 15 years working within the broad field of Imaging Technologies, as both a practitioner and educator, and a significant portion of the previous decade working in and around the field of high-end retouching and key art finishing. If I could use it to do something (with traditional methods) to pixels, it was generally on my radar. But the automated technologies that underpinned these edits were almost entirely foreign to me.
And that shows. Looking back at it now, the language in the note displays a distinct lack of understanding of even the basics of the field. The use of the catch-all ‘algorithm’ was likely a reference to early instances of algorithmic editing I’d seen employed by (maybe) Topaz Labs. I shift from referring to ‘software’ to ‘algorithm’ to ‘AI’ in four sentences, so I didn’t even really know what or who I was reacting to. But it was a moment when I realised that everything I had been doing prior to that, in both my artworking & imaging practise, as well as my teaching work, was going to be completely changed by research and developments within these fields. Even in its primitive (but still very impressive) state, my mind was already wandering toward a feeling of concern for what was going to happen when technologies like these matured to the point they became a viable way for companies to make money.
So even though I didn’t really understand how the technologies were built or how they worked, the note still displayed an awareness of an issue that has become even more relevant to the challenges of being an artworker in 2024 - whether professional, aspiring or enthusiast - when one considers the current state of image synthesising Generative AI technologies, and their incredibly low barrier to entry and even lower risk investment requirements (easy to pick up, easy to abandon). One of the most common - if not the most impassioned - threads of discourse throughout 2023 must surely have been regarding the moral and ethical objections many artists had to their artwork - and its subsequent reduction to simply ‘data’ - being used to train and fine-tune ever increasingly more capable diffusion models, making their pictorial and representational skills, and experience with their mediums, accessible to those with no experience and no mechanical skills. When one takes into account the sheer scale of the LAION datasets - from 400,000,000 publicly scraped image-caption pairs in 2021, to 5,850,000,000 in 2022 - to an uninformed observer, one not embedded within the silos of academia or R&D, it seemed that in the pursuit of technological discovery, development and innovation, the question of “can we do this?” - both technically and legally - was taking precedence over “should we do this?” - admittedly, a much more humanistic consideration.
Beyond the basic question of ‘who are the people developing these technologies?’, the note acknowledges tension between the value systems of multiple fields. Now to be fair, I genuinely believe that the ideological frameworks that underpin both academically oriented R&D in tech focused STEM fields, and the pursuit of artistic and aesthetic practise, are both predicated, in part, on the act of one doing genuinely meaningful work; work pursued with natural scientific and technological, or creative and aesthetic, curiosity, and driven by the importance of being able to contribute meaningfully - in a way that is meaningful for the individual practitioner - this is after all, an important prerequisite for social stability.
However, there’s an entire field of research dedicated to exploring the benefits of a more productive collaboration between the humanities oriented fields of the Arts with STEM. This is because established STEM models tell you “this is how things are”. Traditional STEM workflows analyse, abstract, count, mark time, plan procedures step-by-step, verbalise and make statements based on logic. Conventional STEM education gives learners facts, answers, knowledge, certainty and truth. This is the kind of thinking that helps one look at human made artwork with an excited, natural curiosity and see output first and foremost, patterns that can be learned by machine-driven data analysis.
"The arts are the most powerful means of strengthening the perceptual component without which productive thinking is impossible in any field of endeavor."
Rudolf Arnheim
And yes, artworking is output. But it is output inextricably linked to a process driven practise. And even for the most jaded of strictly vocational artists, artistic practise is still more than just developing and refining mechanical skills. It’s about visualising and communicating the ideational. It’s about reconciling the relationship between what we see and what we know. It’s about awareness, it’s exploring the personal associations we use to interpret what we look at and create meaning from what we see. It’s about enhancing our ability to recognise and understand visual phenomena, allowing us to utilise a more sophisticated repertoire of tools to interpret perceptual clues and determine aesthetic outcomes. It is in this space that many practitioners develop heightened sensitivities towards more humanistic considerations - the kind that would not only ask “should we do this?”, but also those well equipped to confront and engage with the nuance and complexities of the inevitable follow on questions that would emerge from that initial enquiry.
Now to be fair, from the instant the very concept of Artificial Intelligence existed, there have been countless impassioned voices that have spoken, at length, tirelessly to the point of breaking, warning those pushing development forward of the need for cautious restraint, and critical reasoning for those of us engaging with technological developments. Warning us of the biases inherent in the systems. Questioning the hype. Urging us to slow down, to not let the needs of commerce dictate the pace and methods of their deployment. And right alongside them, there have always been artists working critically with these technologies from the moment they developed and evolved. They raised the issues, they explored the problems, they experimented with the developments in their own work. As the field has grown, and the reach of these technologies impacts more of the general public, it is unsurprising there are even more of them working in this critical capacity today. The idea of an external system wherein the artist cedes partial or total control has always been fruitful grounds for both artistic and philosophical exploration. This relationship between machine-assisted artist and machine-made art has propelled discussions and ideas that focus primarily on systems, their role, their relationship to creativity and authorship and system taxonomies. The reliance on large scale datasets in the production of ML focused synthetic imagery opens up the possibility to discuss and explore visual conventions and biases, to question systems of power and authority and to redefine and democratize AI literacy.
As it stands, many of the artists who’s explorations endured through this early period, and that we now associate with being kind of aesthetic pioneers of these technologies, are the ones who also approached it with the same sense of excited - entirely non-malicious - curiosity often seen embedded in STEM thinking. They were (and still are), intrigued by the idea of pictorial output as data, as patterns that can be learned by machine-driven data analysis. They effectively helped develop the technologies through their explorations, while simultaneously incorporating them into their practise. But this kind of capital a Art can often feel like just another silo - built on language and knowledge frameworks and conceptual considerations that more often than not feel a bit too much like ‘for academics, by academics’. It’s common for those outside the silo to feel frustrated, even demeaned, by the perceptual skill that modern art requires. When confronted with music, dance, theatre, poetry and pictures, often they find themselves in a situation where they literally do not know what to do or which skills to deploy.
While those within make artworks that address and help guide critical conversations, contribute to think tanks and conferences, get involved in policy or ethics or editorial or curatorial roles, from the outside looking in, it can feel like many of these critical insights and concerns don’t really filter out to the general public, in a sort of practical, boots-on-the-ground, day-to-day capacity. Spend five minutes browsing LinkedIn in December 2024 and tell me how many of the current crop of ‘AI Experts’ are wrestling with the same kinds of nuanced, complex philosophical conundrums. And besides, the early adopters co-signing of a 2015 open letter calling for a worldwide ban on autonomous machine intelligence-enabled weapons systems didn’t seem to slow down the more than 800 active AI-related projects in the US military, alongside its request for more than $1.8bn worth of funding for AI in the 2024 budget alone. Their concerns were never going to help Nvidia’s stock value increase by roughly 2,400% over the last five years, transforming them, albeit temporarily, into the world’s most valuable company. The issues they raised were never going to trigger a frenzied wave of venture capital, corporation and government backed investment. Wealth isn’t really accumulated by concerning oneself with the messy, humanistic implications of asking “should we do this?” So I guess we know what won out in the end.
If there’s one thing I believe to be unwaveringly true, it is that unbridled capitalism will always be the loudest voice in the room. It is built on greed, inequality, outrageous profits and absurd incomes. Its vast amount of overreach into our everyday lives helps billionaires and corporations control governments. When these generative technologies were ready for prime time, so to speak, they were always going to be shoved down our throats, regardless of what those engaging with them critically were warning us about, or whether anyone even had a sensible business model ready to implement. History has shown us time and time again that the most significant tech developments and innovations inevitably become disruptive tech products and tech services, and it’s in these changing of the guard moments, where the ideologies embedded in these products and services, that define how they get incorporated into our everyday lives, and the manner in which they impact us the most, also tends to shift significantly.
The tech industry, with its aversion to compartmentalisation and lack of respect for boundaries, its blinkered fascination with disruption-for-profit, with “breaking things, thinking big and moving fast”, has billions of reasons to promote a particular narrative - narratives like ‘anyone can be an artist now!’ or ‘you too can make millions with your brilliant unique ideas!’ and ‘art has finally been democratised!’ It also has the ideal ready-to-deploy-at-a-moments-notice marketing arm of ideologically-aligned aspiring thought-leaders, all self proclaimed domain-experts within weeks of practise, every one of them ready to rage-bait you into engagement mode in their relentless quest for more attention and an invitation to sit at the head of the Influencer table.
And so just like fast food and fast fashion, Silicon Valley was always going to give us fast art: content made quickly (but not cheaply) and consumed in haste, perfect for the relentless content cycle that the tech industry itself helped usher in. Seduced by the capabilities of these technologies, no user having this much fun, gaining immediate access to mechanical artworking skills they didn’t need to develop, thinking they were now a full blown artist - or, to be fair, established artists who were being ground down to powder by increasingly more unrealistic client expectations and higher volumes of lower paying work, simply looking to keep earning in an economy and market that seemed actively hostile towards them - was ever going to ask themselves how nourishing the content they were generating was, or how sustainable the systems that helped create them actually are.
So over the course of 2021-2022, Generative AI technologies fled the silo, having their zeitgeist moment with the public release of Dall-E first, followed by Midjourney and finally Stable Diffusion. And initially, some interesting conversations started happening amongst the rest of us, touching on topics that had been front and centre within academic circles for decades: What did it mean to be an artist? Is machine made art less meaningful than human made art? Can easily reproducible images be considered art at all? If machine made art, generated without the traditional artists input, based only on a machine learning models language-based interpretation of a text input, could generate technically and aesthetically successful images, who actually authored the image? Were ideas the only meaningful component of visual artwork, or was mechanical skill - and the software expertise required to translate those skills - equally, if not more important? It had non artistically inclined people, in their efforts to improve their text conditioning methods, discussing visual thinking strategies and pondering aesthetics, depiction, detection and other common visual literacy concerns.
And then, as could have been easily predicted, the people who had their lifes work scraped - without their permission - reduced to text and image caption pairs and incorporated into datasets that made all of this possible, started interjecting. They began objecting. Loudly. And the people who had found their new favourite toy and felt like they were just having harmless fun generating character sheets for their D&D group felt annoyed. Those who could see a path to leveraging these technologies for their own personal gain felt annoyed. Those who simply wanted to piggy back on the collective skills and experience of artworkers and role play as artists themselves felt annoyed. Those who felt that apparently art had always belonged to only the elite few felt both annoyed, and also validated. Those who had neither the patience or discipline to persist through the messy lifelong learning process of skill development - a pathway present in literally every field of practise - and felt that they should be rewarded for simply having ideas, despite not having any practise in vetting or refining or evaluating these ideas against the ideas of those that had helped to build the medium they seemed to have so little contextual understanding of but also seemed so desperate to be a part of, felt annoyed. Those who didn’t seem to understand that the media they loved, and so desperately wanted to recreate, was created by individuals and teams who had taken the time to develop their skills, refine their visual and aesthetic sensibilities, persist through the exhausting, sometimes deflating and demotivating process of learning software applications that felt at odds with their own innate creative processes, and somehow equated this journey of personal and skill development - again, one present in literally every field of intellectual, aesthetic, artistic, and technical practise (amongst all the others) - as both hostile, elitist gate keeping, and considering their obsession with the ‘democratisation of art’, apparently a form of tyranny.
And this is where the conversation shifted, touching on more hostile threads of discourse, where the tension between the two value systems became much clearer: “This is just a tool, they said the same thing about Photoshop”, “why has art been gatekept for so long?”, “why are all these artists such doomers and laggards and luddites?”, “why aren’t artists happy that more people can now make art?”, “actually, you probably don’t realise but the process of training an ML model is no different to artists learning their craft by drawing from references they encountered in their learning and professional practise!”, “it’s not illegal!” - it’s always a good sign when someone feels compelled to point this out - “finally, people with disabilities can now make art too, which they have never been able to do before I presume!”, “the cat/genie/toothpaste is out of the bag/bottle/tube now (there was a lot of argument by button-pushing going on), there’s no stopping progress, it’s here to stay, artists need to learn to live with it or else they’ll be left behind!”, “why are artists annoyed that their work is being used to train these models, they should be grateful people are even paying attention to their art! I never even knew their work before and now I do, if they don’t want attention what do they even want?”, “I would personally be happy to contribute my work to these models”, “hey [insert name of specific artist who voiced their concerns loudly on various platforms], suck shit, I just finetuned a LoRA on all of your work from your Insta feed because you’re a privileged, elitist, gatekeeping sack of garbage and I hope you spend the rest of your life being increasingly valued out of the market until you become jobless and homeless!”. They said that the only reason artists struggle to make money is because they simply didn’t know how to market themselves. That this was no different to horses vs automobiles, or painting vs photography. That the endgame of potentially compensating artists for use of their work in datasets was a “pay per thought” society. That artists brought this on themselves by being so protective of their skills and they only had themselves to blame. That artists were blockers to progress, enemies of an open, democratic society. There was an overwhelming sense of hostility online in these conversations, directed towards those who were most immediately affected by it, in a sort of “lets just screw over everybody who we can grind between the gears of our APIs because haha, they cant stop us!” manner.
One of my favourite responses to the oft-repeated ‘horses vs automobiles’ analogy, was something along the lines of: this isn’t horses vs automobiles, it’s horses vs a motor that grinds up horses to generate power for automobiles
Again, for emphasis: if there’s one thing I believe to be unwaveringly true, it is that unbridled capitalism - and the values that underpin it - will always be the loudest voice in the room. I know that’s cynical, it’s just where I’ve landed on the issue. Regardless of whether it was a net gain or loss for our culture and our industries, and for us as individuals and the society we all share - who gets to decide that anyways? - corporations could now collect even more delicately intertwined threads of incredibly personal, immensely valuable user data and secure extraordinarily inflated amounts of investment funding regardless of how much energy it was going to burn in the process, or how getting it to scale was going to rely on exploiting outsourced, marginalised workers in poorer nations, or how unleashing disruptive, unregulated technologies that governments and lawmakers were not remotely prepared for, technologies that made it effortless for predators to produce non-consensual, photorealistic deepfakes to sexually abuse their teenage classmates - yes, they are all separate links - might not actually be a great thing, and might require some care and caution. No time for that! Think big, move fast, break things!
It was clear that this was going to exist. Even though 18 months later, with no real sign of a compelling, sustainable, profitable business model in sight, there are still so many viable open source options already available, and developments on those will continue for as long as there is interest. Even if investment on the commercial side were to immediately cease and all progress ground to a halt tomorrow, there are more than enough tools available now that can be run locally on consumer hardware that can perform well enough for many. This wasn’t going away. And in the process, it was going to bring with it a lot of values - embedded within its systems and espoused loudly by its boosters - that felt very at odds with those that its established practitioners were likely most familiar with. It was going to continue to develop and evolve - at a frantic pace - it was going to be used and adopted by A LOT of people, because it was fun and so much more accessible for the general public than NFTs and blockchains and VR/AR/Web3. And with that surge of widespread adoption, with its gamificiation (and subsequent lootboxification) of art making, it has the potential to impact entire fields of learning and skill development and value making and employment. The tech industry had been pushing the creative industry - particularly independent artists - to a slow and steady breaking point for three decades, but they had never gone this big before. Whether artworkers adopted or ignored the technologies partially or completely, this was absolutely not going away, and it was going to drastically change the landscape for what it meant to be, and aspire to be, an artist working in a visual medium.
It was noisy, messy and very, very nasty. And 18 months later, it still is. The justifiable emotions displayed tended to push those aforementioned specific issues to the front. But beneath the noise floor, alongside the moral and ethical and legal considerations, I found myself thinking about the downstream impact of these developments from a professional perspective: questioning how these technologies would change how my students thought of making art, and how they approached it in the early stages of their careers, and what it would be like to enter the industry with these technologies being fully embraced. How successful would these technologies become in establishing themselves as an ‘industry standard’? What impact would lowering the skill barrier to entry within the field have on artworkers ability to earn, and the development of artworking techniques, practises and methodologies? How would it change how we thought about pictorial images? How would it shape our aesthetic sensibilities? How would it alter the way we approached, talked about and understood the creative process? How would it shift both learning and career trajectories for student and aspiring professional artists? Could it gut The Creative Middle and leave a huge knowledge and skill gap in the mid to long term future? Would it usher in a full blown race to the bottom in terms of quality for commercial artworking? Would its fascination with both speed and output over process atrophy some of our most valuable, human-centred practises?
And so here we are. It’s nearly seven years to the day since I wrote that note. I’ve spent the last two years working with synthetic data in an artworking, pipeline, R&D and training capacity, primarily for feature film VFX, while observing the conversations online with equal parts frustration, horror and anger. In that time, from the perspective my vocational role has afforded me, in my capacity as both a lifelong independent artworker and educator, I’ve gained a decent amount of insight into how the landscape might shift for professional, aspiring and enthusiast artworkers in a post-Generative AI world. On the challenges visual storytellers will encounter imagining, creating, learning and earning through the latent space. What practical impacts machine learning technologies will have on creative vocations, artistic practise, aesthetic sensibilities and art & design education. I’ve been keeping notes, exploring, developing and thinking through ideas, and waiting until I had the right amount of time to begin sharing these notes with others - notes intended as a touchstone for aspiring, enthusiast and established artists in the future who might choose to engage with, adopt, resist, oppose, refuse, or otherwise simply exist within, a culture increasingly dominated by Generative AI technologies and their associated output.
This feels like a good place to begin.