Graphical style transfer
The most basic kind of style transfer is a transfer of colors. Basically, the AI is given a long list of pictures in color and in black-and-white, until it learns to turn a black-and-white picture into color. This then gives an AI that can do colorization:
Notice the flickering in the video. That is due to lack of "temporal coherence". Basically, the AI is coloring frame-by-frame, not knowing that the same region in frame 12 and frame 13 should be colored same.
The following video shows artistic style transfer with temporal coherence:
This more recent one can do multiple style transfers on the same image, using mask-layers!
Somebody gotta try Style 1 (pile of skulls) on Hitler!
Some style transfers are... literally nuts:
Those are the research level stuff. EbSynth is a production-ready software that can do style transfer, now!
IBM has released a demo tool, GANPaint, which can be tried now!
The following AI is probably the most directly useful for artists:
This is also very useful for retouching photos.
This is the best. Draw something crudely and it instantly turns into realistic photo.
Enhancement
Here, instead of training the AI to transfer between styles, the AI is trained to turn a low-quality picture into high-quality.
It can do smoothing, tweening, and other kinds of tedious interpolation work!
It can learn to fill in the blanks, automatic Photoshopping:
And it can do... 3D ken burns?
This last one is not obviously useful for art, but it just seems important somehow. I'm sure there is an art project hiding there, somewhere.
Digitize physical things
Physical object -> one photo -> 3D model
This one is going to be big: everyone is going to have some VR avatars!
Motion tracking: video -> 3D skeleton of the pose
Deep fake and other kinds of style-transfer puppetry
This is a more radical kind of style transfer. Basically,
Style + Content = Art
where the "style" can be anything.
Pose:
Face features:
Time-of-day and season:
Caricature:
Words:
I would vote for Greta Trump!
Voice:
Putting a texture on an uneven surface, like putting Kandinsky the Stanford Bunny:
And photoshopping videos:
Creativity with a mildly random AI partner
AI music has a long history. The recent MuseNet is a great thing to play with.
Fursona generator.
And of course you can go full on random and make absurdist humor by injecting AI-generated nonsense at each turn, which the Botnik has accomplished very well.
There's a blog called AIWeirdness that's all about using AI to generate semi-random absurd things. My favorite is when it makes My Little Pony names.
Here's a tip for getting started: find an online implementation of GPT-2, such as TalkToTransformer, and put in your favorite fanfic opening, and see it work its magic...
More realistic animation
Realistic animation requires not just correct light-and-shadow, but also correct motion. Both for quadrupeds and for bipeds, the technology is ready. No more animation cycles and specifying the angle of each joint. Just draw the path and it will walk it realistically!
Others
No idea what to call it, other than an attempt to break the spacetime continuuuuum.
Again, not sure what to call it, but geometric algebra + precise woodwork + 3D printing = natural magic.
Computational hydrographic printing is the perfect complement to 3D printing!
Computational origami is a thing now! And Robert Lang is the master
There's also curved-folding, which was initially studied by Huffman, as early as 1976 (Curvature and Creases: A Primer on Paper). Yes, that Huffman who is famous for the Huffman code.
And here's a curved origami with strings attached!
For more cool pictures, see Curved Folding (ACM SIGGRAPH 2008). To play with computational origami yourself, try this OrigamiSimulator.
There's also curved-folding, which was initially studied by Huffman, as early as 1976 (Curvature and Creases: A Primer on Paper). Yes, that Huffman who is famous for the Huffman code.
And here's a curved origami with strings attached!
For more cool pictures, see Curved Folding (ACM SIGGRAPH 2008). To play with computational origami yourself, try this OrigamiSimulator.
CUTTING DUCKS IS UNDESIRABLE
NOW THE DUCK IS HAPPY!
I have 0 idea why you'd want to cubify anything, but it's funny!
This tool can separate multiple voices that are mixed together, which could be useful for video post-processing.
A somewhat niche application that could be useful for 3D artists: synthesizing the visual qualities of new materials
A clever trick to make VR more immersive:
The panoptic studio: takes a full perspective of the scene.
Robot-dancing, now with real robots:
... robot remix?
Fail compilations isn't very artistic, but still
And finally, there's the very opposite of high-tech...
No comments:
Post a Comment