M. Seth Jones
Synthetic Media specialist
, consulting in video, image and speech synthesis via generative and diffusion models. With an emphasis on photorealistic face swaps and natural-sounding voice conversion, I curate, caption, train and fine-tune custom datasets to deliver visually striking and highly realistic outputs across creative media applications.I collaborate closely with VFX supervisors, producers, artists and other stakeholders to develop custom pipelines that improve temporal coherence, enhance detail and correct artifacts. With an exhaustive knowledge of Stable Diffusion, ComfyUI and RVC, I utilise sophisticated techniques and approaches to optimise fine control over latent outputs, employing a variety of machine learning tools and methodologies to generate desired results during inference.
Across both enterprise and higher education, I've designed and facilitated strategic learning outcomes for the likes of BBC Studios, Ogilvy Australia, RMIT University, Adelaide University, Australian Centre for Photography, ABC Australia, Røde Microphones, The Monkeys, The Royals and Carnival Studio. With over 20 years production experience in key visual artworking, photo compositing and high-end beauty and portrait retouching, I have produced key artwork for Disney+, Sony Pictures, Adidas, Toyota, Lionsgate, The Refinery Creative, IFC Films, Audi, Honda and McLaren.
A curious and passionate observer with a lengthy and varied career in creative digital media, I have developed a unique artistic knowledge of machine learning tools and methodologies. By applying these novel and innovative techniques in unconventional ways, I'm able to generate new insights and solutions, designing experiences that engage users on emotional, cognitive, and aesthetic levels. With a comprehensive understanding of various ML frameworks, tools, and approaches, I combine artistic expression with technological innovation and scientific curiosity to solve complex problems, and explore new frontiers at the intersection of art and machine learning.
Contact
As of 05/24, I am currently unavailable for additional consultant engagements. If your organisation is looking for specialist advice in the field of synthetic media or advice on implementing Stable Diffusion, ComfyUI or RVC into your creative pipelines, feel free to connect with me on LinkedIn, or get in touch via email.
Current experience
Metaphysic
VFX Trainer and Facilitator (Synthetic Media)
May 2024 - Present
Facilitation and Learning Design expert, developing and delivering tailored training programs to creative teams on synthetic media tools in visual effects production for film. Collaborating with VFX supervisors, synthetic media leads and experts, I help identify training needs, design learning frameworks and develop strategies for improvement. Duties include designing and developing comprehensive training programs, creating engaging learning materials, delivering video training sessions, collaborating with department heads, maintaining comprehensive documentation, and curating resource repositories.
Consultant
Creative Technologist (Synthetic Media)
Jan 2023 - Present
Creative Technologist specialising in Synthetic Media, consulting on the implementation of cutting-edge generative video, image and speech synthesis technologies like Stable Diffusion, ComfyUI and RVC into VFX, film, creative and experiential media. I help build complex synthetic media productions that push the boundaries of photorealistic face-swaps and natural sounding voice cloning while utilising workflows that maintain efficiency, adaptability and collaboration.
- Curating and preprocessing high-quality datasets of human faces or relevant subjects, including artifact removal, data normalization, and diversity-enhancing augmentations
- Developing detailed dataset captions using techniques like CLIP Interrogation, providing model input prompts that condition on specific features to achieve desired output
- Constructing data preprocessing pipelines using Python and libraries like NumPy, Pandas, and OpenCV, incorporating techniques like data cleaning, augmentation, and normalization to prepare high-quality datasets for training
- Selecting suitable generative models or diffusion models, customising architectures, and training them on prepared datasets, including fine-tuning pre-trained models for specific use cases
- Training custom SD 1.X/2.X/XL checkpoints, LoRAs and embeddings to enhance model performance in generating photorealistic faces or natural-sounding voices, accurately mimicking real-world data
- Leveraging technqiues such as sampling strategies, conditioning methods, noise schedules and multi-modal inputs, alongside advanced technologies like CLIP/ViT-L/ViT-H, inpainting, segmentation masks, Multi-ControlNet, IP-Adapter and style transfer for granular latent space control, enhancing output at the inference stage to generate high-fidelity synthetic media
- Designing and implementing efficient synthetic media generation pipelines, integrating various generative models, tools, and techniques into cohesive workflows. Collaborating with VFX or film production teams to develop custom solutions as needed. Developing rapid prototypes, testing in various scenarios, gathering feedback, and iterating on models or pipelines for continuous improvement
- Developing custom utilities and helper functions in Python, streamlining workflows by creating data loading/saving tools, metrics calculation, and model evaluation functions to enhance efficiency. Rapidly prototyping new ideas and techniques, creating proof-of-concept code or experimenting with various algorithms and architectures
- Evaluating and validating model performance through ongoing assessment of generated outputs, refining models via hyperparameter adjustments, regularization techniques or incorporating extra data for continuous improvement. Optimising model performance by fine-tuning, adjusting training parameters, and utilising GPU processing for efficient training and high-quality output
- Maintaining a critical perspective on results, consistently prioritizing high-quality outputs while retaining depictive, detective and identity adherence
- Remaining informed on advancements in cutting-edge and emerging generative models and machine learning tools and techniques, applying up-to-date knowledge and methodologies and experimenting with them to discover novel applications in synthetic media production
- Reflecting on ethical implications of synthetic media projects, considering potential biases or unintended consequences, and integrating ethical considerations into project development strategies. Maintaining awareness of legal guidelines and regulations concerning synthetic media usage in film productions, ensuring all projects comply with laws and industry best practices for rights management, consent, and attribution.
Recent experience
Metaphysic
Synthetic Media Specialist | Stable Diffusion Artworker
Apr 2023 - Dec 2023
Synthetic Media expert, specialising in hyperreal image, audio and speech synthesis via generative models.
- Training and optimising DL/ML models to generate hyperreal synthetic media, employing a strong emphasis on generative models and their creative potential
- Running generative models on both stills and video footage, optimizing parameters to ensure outstanding visual fidelity
- Implementing and developing pipelines to effectively curate and refine generated data to enhance realism and aesthetics
- Leveraging Stable Diffusion methodology to generate photorealistic results, employing fine-tuning techniques on 1.X & 2.X models using custom datasets
- Demonstrating a comprehensive understanding of dataset captioning and model training hyper-parameters, alongside their impact on model performance
- Utilising settings, extensions, prompting, in-painting and loopback techniques to achieve desired results during inference, while maintaining a meticulous focus on temporal coherence, detail enhancement, and artifact correction
- Collaborating closely with the data, model and broader creative teams, seamlessly integrating generated outputs into existing pipelines and contributing to the creation of new workflows
- Possessing a discerning eye for dataset refinement, effectively adapting data to elevate the quality of specific results
- Successfully managing and training multiple models concurrently, ensuring efficient processes and optimal outcomes
- Maintaining a critical perspective on results, consistently prioritizing high-quality outputs while retaining depictive, detective and identity adherence
- Facilitating versatile and efficient workflow and pipeline implementation and integration with a proficiency in both Python and PyTorch
- Utilising extensive professional experience in facial, portrait and beauty retouching work to enhance fidelity of generated artwork.Synthetic Data expert and Stable Diffusion artworker specialising in hyperreal synthetic media and generative models, with an emphasis on photorealism.
Past experience
Freelance
Creative Artworker and Retoucher
2002 - 2022
Clients include: Disney+, Sony Pictures, Adidas, Toyota, Lionsgate, The Refinery Creative, IFC Films, Audi, Rolex Mentor & Arts Initiative, Schwarzkopf, Honda, McLaren, Guess, Akcelo, Gemba Group, ASOS, TRP Agency, DSF, Carnival Studio, Curzon Film, Tourism Australia, Transmission Films, Velvet Pictures, Elle Magazine, GQ Australia, Goulburn Valley, Masterfoods, Wrangler, EVO Hair, Ceremonia, Mash Design, asap Skin Products, Emily Abay, Rachel Freire, Nathan Gallagher, Lara Jade, Hannah Khymych, Andrew Babarczy, Nat Lanyon, Adrian Mesko, Zac Stone, Amber Toms.
- Providing end-to-end finished key artworking, creative retouching, digital illustration and photographic compositing for static and digital OOH campaigns and streaming and theatrical key art media.
Freelance
Imaging Technologies Facilitator & Learning Designer
2008 - 2022
Clients include: Ogilvy Australia, BBC Studios, The Monkeys, The Royals, ABC Australia, Carnival Studio, Røde Microphones, MYER, Studio One20, Hips & Curves, Larsen&Talbert, Madara Cosmetics, asap Skin Products, Jan Wischermann, and Snapcomm Fotográfico.
- Used advanced program and course level pedagogical design practises to deliver strategic learning outcomes, developing and delivering over 2000 tailored blended imaging technologies sessions to more than 1000 students from 35 different countries.
- Applying expert finished art knowledge and creative retouching skills to design and deliver face-to-face and remote lectures, seminars and instructional materials in line with the strategic direction of organisations production goals.
- Using specialist knowledge in ML image making tools to advise on and develop responsible, artist-centred implementations of tools such as Stable Diffusion into creative pipelines.
University of Adelaide
Digital Photomedia Lecturer and Facilitator
2019 - 20
- Developed and facilitated face-to-face workshops and online instructional materials for effective learning outcomes for Bachelor of Photography program.
- Assessed student work, graded achievement criteria and provided feedback on results.
RMIT University
Digital Photomedia Lecturer and Facilitator
2010 - 11, 2015
- Designed and facilitated the 78 hour, 26 week core Imaging Technologies program to more than 100 Bachelor of Photography students.
- Applied pedagogical expertise to program, course and activity educational design to initiate and develop a new online component of the course.
- Maintained knowledge and examples of current online and blended pedagogies used across the University, including successes and failures and evaluations of usage.
- Provided advice to academics on learning technology resource development projects, including managing expectations about project scope, timelines and potential outcomes.
Australian Centre for Photography
Digital Photomedia Lecturer and Facilitator
2013 - 14
- Created and delivered a series of 48 hour, 16 week workshops and seminars on advanced photographic post-processing and post-visualisation.
Photography Studies College
Digital Photomedia Learning Designer
2014
- Designed a 39-hour, 13 week core Tier 3 and Tier 5 subject guide for the Bachelor of Photography program.
City Desktop Training
Creative Artworking Lecturer and Facilitator
2012 - 13
- Delivered intensive, hands-on workshops on advanced, intermediate and foundation photographic post-processing and post-visualisation.
Fearless Media
Creative Artworking Lecturer and Facilitator
2010 - 11
- Provided advanced, intermediate and foundation Adobe Creative Cloud suite application techniques to corporate clients.
Skills
Stable Diffusion
Expert / 2+ years
ComfyUI
Expert / 1.5+ years
Adobe Photoshop
Expert / 22 years
Adobe Illustrator
Expert / 21 years
Adobe Lightroom
Expert / 14 years
Phase One Capture One
Expert / 13 years
Adobe Premiere Pro
Expert / 12 years
Adobe After Effects
Expert / 12 years
Adobe InDesign
Expert / 12 years
Blackboard Learn
Advanced / 11 years
Adobe Animate
Advanced / 10 years
Procreate
Advanced / 10 years
Blackboard Collaborate
Advanced / 7 years
Adobe Portfolio
Advanced / 4 years
Adobe XD
Advanced / 4 years
Moodle
Advanced / 3 years
Current experience
Metaphysic
VFX Trainer and Facilitator (Synthetic Media)
May 2024 - Present
Facilitation and Learning Design expert, developing and delivering tailored training programs to creative teams on synthetic media tools in visual effects production for film. Collaborating with VFX supervisors, synthetic media leads and experts, I help identify training needs, design learning frameworks and develop strategies for improvement. Duties include designing and developing comprehensive training programs, creating engaging learning materials, delivering video training sessions, collaborating with department heads, maintaining comprehensive documentation, and curating resource repositories.
Consultant
Creative Technologist (Synthetic Media)
Jan 2023 - Present
Creative Technologist specialising in Synthetic Media, consulting on the implementation of cutting-edge generative video, image and speech synthesis technologies like Stable Diffusion, ComfyUI and RVC into VFX, film, creative and experiential media. I help build complex synthetic media productions that push the boundaries of photorealistic face-swaps and natural sounding voice cloning while utilising workflows that maintain efficiency, adaptability and collaboration.
- Curating and preprocessing high-quality datasets of human faces or relevant subjects, including artifact removal, data normalization, and diversity-enhancing augmentations
- Developing detailed dataset captions using techniques like CLIP Interrogation, providing model input prompts that condition on specific features to achieve desired output
- Constructing data preprocessing pipelines using Python and libraries like NumPy, Pandas, and OpenCV, incorporating techniques like data cleaning, augmentation, and normalization to prepare high-quality datasets for training
- Selecting suitable generative models or diffusion models, customising architectures, and training them on prepared datasets, including fine-tuning pre-trained models for specific use cases
- Training custom SD 1.X/2.X/XL checkpoints, LoRAs and embeddings to enhance model performance in generating photorealistic faces or natural-sounding voices, accurately mimicking real-world data
- Leveraging technqiues such as sampling strategies, conditioning methods, noise schedules and multi-modal inputs, alongside advanced technologies like CLIP/ViT-L/ViT-H, inpainting, segmentation masks, Multi-ControlNet, IP-Adapter and style transfer for granular latent space control, enhancing output at the inference stage to generate high-fidelity synthetic media
- Designing and implementing efficient synthetic media generation pipelines, integrating various generative models, tools, and techniques into cohesive workflows. Collaborating with VFX or film production teams to develop custom solutions as needed. Developing rapid prototypes, testing in various scenarios, gathering feedback, and iterating on models or pipelines for continuous improvement
- Developing custom utilities and helper functions in Python, streamlining workflows by creating data loading/saving tools, metrics calculation, and model evaluation functions to enhance efficiency. Rapidly prototyping new ideas and techniques, creating proof-of-concept code or experimenting with various algorithms and architectures
- Evaluating and validating model performance through ongoing assessment of generated outputs, refining models via hyperparameter adjustments, regularization techniques or incorporating extra data for continuous improvement. Optimising model performance by fine-tuning, adjusting training parameters, and utilising GPU processing for efficient training and high-quality output
- Maintaining a critical perspective on results, consistently prioritizing high-quality outputs while retaining depictive, detective and identity adherence
- Remaining informed on advancements in cutting-edge and emerging generative models and machine learning tools and techniques, applying up-to-date knowledge and methodologies and experimenting with them to discover novel applications in synthetic media production
- Reflecting on ethical implications of synthetic media projects, considering potential biases or unintended consequences, and integrating ethical considerations into project development strategies. Maintaining awareness of legal guidelines and regulations concerning synthetic media usage in film productions, ensuring all projects comply with laws and industry best practices for rights management, consent, and attribution.
Recent experience
Metaphysic
Synthetic Media Specialist | Stable Diffusion Artworker
Apr 2023 - Dec 2023
Synthetic Media expert, specialising in hyperreal image, audio and speech synthesis via generative models.
- Training and optimising DL/ML models to generate hyperreal synthetic media, employing a strong emphasis on generative models and their creative potential
- Running generative models on both stills and video footage, optimizing parameters to ensure outstanding visual fidelity
- Implementing and developing pipelines to effectively curate and refine generated data to enhance realism and aesthetics
- Leveraging Stable Diffusion methodology to generate photorealistic results, employing fine-tuning techniques on 1.X & 2.X models using custom datasets
- Demonstrating a comprehensive understanding of dataset captioning and model training hyper-parameters, alongside their impact on model performance
- Utilising settings, extensions, prompting, in-painting and loopback techniques to achieve desired results during inference, while maintaining a meticulous focus on temporal coherence, detail enhancement, and artifact correction
- Collaborating closely with the data, model and broader creative teams, seamlessly integrating generated outputs into existing pipelines and contributing to the creation of new workflows
- Possessing a discerning eye for dataset refinement, effectively adapting data to elevate the quality of specific results
- Successfully managing and training multiple models concurrently, ensuring efficient processes and optimal outcomes
- Maintaining a critical perspective on results, consistently prioritizing high-quality outputs while retaining depictive, detective and identity adherence
- Facilitating versatile and efficient workflow and pipeline implementation and integration with a proficiency in both Python and PyTorch
- Utilising extensive professional experience in facial, portrait and beauty retouching work to enhance fidelity of generated artwork.Synthetic Data expert and Stable Diffusion artworker specialising in hyperreal synthetic media and generative models, with an emphasis on photorealism.
Selected work
BBC Studios
Video Training and Workshops
47 videos / 32 hours / 8 workshops
BBC Studios broadcasts 11 international channels, including BBC AMERICA and BBC Canada. Their creative team oversees the production art of numerous global media properties. Their work is seen across multiple platforms by millions of people. I designed and facilitated a 32 hour video training program for their production artists to ensure the highest standard of finished output.
RMIT University
Blended Facilitation and Assessment
32 tutorials / 96 hours / 28 webinars / 7 assessments
Founded in 1887 and ranked 16th in the world for art and design, RMIT is the leading art and design school in the Southern Hemisphere. I designed and facilitated the 78 hour, 26 week core Imaging Technologies program across two separate cohorts to more than 100 students, including instigating and developing an all new online component of the course.
Ogilvy Australia
Standalone Workshops and Webinars
6 sessions / 8 hours / 1 ebook
Ogilvy Worldwide is one of the largest marketing communications networks in the world, currently employing over 10,000 people across 450 international offices and 120 countries. Founded in 1977, Ogilvy Australia's clients include IBM, Coca-Cola, KFC and American Express. I was hired to design and and facilitate eight hours of training to senior Creative Directors.
University of Adelaide
Face-to-face Facilitation and Assessment
39 tutorials / 195 hours / 9 assessments
Established in 1874, the University of Adelaide is the third-oldest university in Australia, and is a member of the Group of Eight, comprised of Australia's leading research-intensive universities. Across three separate cohorts, I designed and facilitated 195 hours of face-to-face workshops and tutorials, to ensure successful learning outcomes for first and second year students of the Bachelor of Photography program.
The Royals
Screencast and Video Training
16 screencasts / 24 hours / 6 walkthroughs
The Royals are an agency fuelled by passion and curiosity, propelled by thinkers and tinkerers, creatives and coders, hustlers and hackers. Their clients include Mercedes-Benz, the National Gallery of Victoria, Spotify and Coopers Brewery. I was called on to design and facilitate 24 hours of pre-recorded and real-time video and screencast training for their finished art team.
The Monkeys
Screencast and Webinar Training
8 screencasts / 18 hours / 12 walkthroughs
Founded in 2006, The Monkeys is a creative agency based in Sydney, Australia, working at the intersection of advertising, entertainment and technology. I was brought on to design and facilitate an 18 hour real time, screencast walkthrough training program for a dozen designers, artists and senior creative directors.
Røde Microphones
Screencast and Video Training
18 videos / 12 hours / 8 screencasts
Founded in 1967, Røde is a world leading audio technology company that designs and manufactures audio and recording hardware and software across consumer, prosumer and professional markets. I was hired to design and facilitate 12 hours of real time screencast and video training for their graphic design team.
Carnival Studio
Screencast and Video Training
51 videos / 32 hours / 12 walkthroughs
Established in 2002, Carnival Studio is a boutique graphic design company specialising in developing key art, branding and immersive marketing campaigns for film, television and home entertainment clients. I designed and facilitated 32 hours of screencast and video training for their senior creative director, focusing on advanced retouching workflows to address the unique production goals of theatrical and streaming key art.
ABC Australia
Screencast and Video Training
56 videos / 26 hours / 8 walkthroughs
ABC Australia is Australia's international television service, delivering a mix of programs including news and current affairs, sport, comedies, drama, documentaries, education and award-winning children's programs. Their key art is seen by millions in 40 countries across the Asia-Pacific region. I designed and facilitated 26 hours of video training to help their senior graphic design team achieve higher standards of finished artwork.
Australian Centre for Photography
Face-to-face Facilitation
16 tutorials / 48 hours / 6 ebooks
Founded in 1973, the ACP is the leading Australian organisation dedicated to photography and new media. I designed and facilitated a series of 48 hour, 16 week workshops for their staff and students.
Photography Studies College
Program Design
13 tutorials / 39 hours / 3 assessments
Established in 1973, PSC is a multi award winning tertiary photography college located in Melbourne. I designed a 39 hour, 13 week core Tier 5 subject guide for the Bachelor of Photography program.
Studio One20
Standalone Workshops and Webinars
3 workshops / 24 hours / 3 ebooks
One20 is a content production company that specialises in the execution of photography, video, design, print and other complementary creative and production services. Based in Melbourne and Sydney One20 produces work for some of Australia's most loved brands such as AAMI, Bonds, BMW, Coke, KFC and Myer. I designed and facilitated 24 hours of training workshops to its production team.
asap Skin Products
Screencast and Video Training
28 videos / 16 hours / 8 walkthroughs
asap is Australia's number one cosmeceutical skin care range. I designed and facilitated 16 hours of training to their design and production team.
Madara Cosmetics
Video Training and Workshops
12 videos / 16 hours / 8 workshops
MADARA is a Latvian manufacturer of organic skin care products. I designed and facilitated 16 hours of training for their art directors.
Hips & Curves
Video Training and Workshops
12 videos / 24 hours / 12 workshops
Located in Southern California, Hips & Curves has been providing luxury plus size lingerie that celebrates the beauty and sensuality of fuller figures since 2000. I designed and facilitated a 24 hour training program for their senior art team.
Larsen&Talbert
Video Training and Workshops
12 videos / 12 hours / 6 workshops
Larsen&Talbert's unique brand of celebrity portraiture has featured in magazines such as Esquire, InStyle, Entertainment Weekly, Playboy and People. Their commercial clients include Bravo, Hallmark Channel, Fox Broadcasting, Disney, Turner Broadcasting and NBC Universal. I designed and facilitated 12 hours of workshops to their production team.
Jan Wischermann
Video Training and Workshops
28 videos / 36 hours / 18 workshops
Based in Düsseldorf, Jan Wischermann offers professional high-end retouching and is a post-production specialist, specialising in creative image processing of beauty, fashion and commercial productions. Across two distinct courses, separated by multiple years, I designed and facilitated 36 hours of training to Jan covering high-end beauty and portrait retouching, editorial aesthetics, lighting, colour grading and post-processing. Additionally, I designed multiple unique colour analysis and grading workflows for him to use in his work.