A few years ago, I worked at several early stage startups involved in consumer facing tech, where it was apparent that problems in these spaces couldn’t be solved through tech alone, but required a confluence between tech, creativity, user-centric design and domain specific insight in order to build something meaningful to the user.
It was during this time that I struck up an association with Nik Lodge. Nik is a creative wizard. He has a formal background in design and photography, and has held several creative director positions. What really struck me about Nik is his ability to wrap his creative mindset around tech concepts, and devise clever ways to marry the two sides. Over several years, we had loads of fun brainstorming ways of using AI/ML to optimise creative products, and (perhaps less intuitively) doing the opposite as well, namely using creative tools to improve AI/ML methods. I’ve summarised some of these ideas below, as examples of what could be doable if you have technical and creative people collaborating effectively.
Using AI/ML to enhance creative products
Nik and I spent time contributing to the development of fashion tech tools at a couple of startups. One of our early projects together, along with YJ Kim and Javier Rodriguez Zaurin, was to use a fashion clothing profiling tool to identify trends in a new collection of clothing from New York Fashion Week, and publish semi-automated summarised reports of the key trends. The objective for the design was to make complex data visually accessible and understandable to a mass audience by illustrating colour trends taken from the fashion show imagery pixel data. By combining the data output with actual clothing in the design it was easy for anyone to see not only the predominant colour trends, but also themes in colour combinations. This was just a simple prototype, but with some additional breakthroughs on the computer vision side (which we achieved a few years later in a different company and context), this could have led to detailed, accessible and automatically generated data driven reports on the state of different areas of consumer fashion, with regards to many key clothing properties beyond colour.
Other projects we brainstormed included a user interface for fashion recommendations that built up a user’s persona based on their choices and lists that they created in a similar way to how Spotify functions, an app that assessed a person’s current mood in order to suggest entertainment and products that would improve their frame of mind as well as tap into their aspirations, and a project that would help creatives build mood boards around a brief using AI to power their creation.
Using creative tools to improve AI/ML
It’s less obvious to see how creative tools can make an impact on the highly technical realm of AI/ML algorithm development, but there’s a lot of space for innovation in this direction too.
A few years ago, in collaboration with Molshree Vaid, we spent time brainstorming a concept for a smart UI for providing a user with clothing recommendations, which would collect feedback from them about whether certain items were a good fit or not (and if not, why), and use this to refine the recommendations. This would have gone beyond what we knew about the state of fashion recommendation systems at the time, which often did single shot profiling and recommendation, or would recommmend on the basis of criteria of uncertain relevance (e.g. if you were interested in product X, then you would be shown products that others who viewed X have previously bought). The idea of actively evolving the recommendations through the user’s input, being open about how the algorithm perceived the user (which is an important concept these days, given numerous concerns about the presence and impact of biases in machine learning models), and allowing the user to actively challenge those perceptions, was key to taking this UI concept a step towards potentially being able to power a more engaging, transparent and personalised AI/ML based recommendation system.
Another example is when Nik saw an opportunity whilst I was developing a method to automatically cut out (or “segment”) fashion clothing from fashion model photos. A key requirement in the development of these algorithms is having access to training data, where the algorithm is presented with a set of inputs comprising the original photo, and a black-and-white image (or “binary mask”) showing the garment of interest having been filled in, so the algorithm can learn where the boundary of the clothing is. The masks are often provided by manual annotators, and getting them to produce these binary masks is time consuming and expensive, since they have to carefully draw around each item of clothing in Photoshop. We were in a cash-strapped early stage startup when we did this, and our algorithm’s performance at the time had plateaued, since we couldn’t afford the requisite quantity of manually labelled data to make it better. Nik’s insight was to take videos of fashion models on a green screen set, where the camera would gradually change position, and the model would adopt different poses. The binary masks would be created in Adode After Effects using tools to adapt a mask from one frame to the next. This would still require manual effort, but it would be significantly quicker than drawing masks from scratch, and would allow us to amplify the amount of training data that we could build on our fixed budget. By doing this for a range of garment types, lighting conditions and skin tones, we could have, in theory, built up a training set that would complemented our efforts to manually label existing fashion photos, which would have allowed us to improve the power of our fledgling computer vision algorithms.
The value of “creatives who get tech” to innovation in consumer-facing tech
25 years ago, Clayton Christensen wrote about disruptive innovation, where innovation can yield opportunities for growth and disruption of existing markets. When it comes to consumer-facing tech, people like Nik, a new breed of “creatives who get tech”, can be extremely valuable in finding this innovation edge, particularly when they’re paired up with someone from the technical side and given free rein to collaborate. Together, they can identify opportunities to provide significant value-adds that exceed those that can be provided through tech alone.
If this sounds like what you need, and you’re looking for your own “creative who gets tech”, Nik is currently open to new opportunities, and would welcome discussions with anyone who would like to get his creative input on their problems. If you have any thoughts on this idea, or would like to learn more about what we did, please feel free to get in touch with either of us. We would also love to hear about similar experiences about how other “creatives who get tech” have helped to drive innovation. (It would be great to do a follow up post that expands on this theme with examples from other people.)
[1] Original photo is from https://www.pexels.com/photo/attractive-beauty-catwalk-clothes-262039/, and is under a CC0 license. Edits were made to the photo by Nik to reflect some of the fashion tech that we worked on.