EP12: Synthetic Yet Real - The New Age of Personalized Video Content ft Ashray Malhotra

Can we build a camera within a toothbrush? Crazy smart, right? Like they're just born smart. You do anything. You put them in any situation, they will be the smartest person in the room. >> You're one of them. >> I don't think I'm one of them. >> Your intent was always to come to a place to convert text into movies. >> It all sounds pretty simple today, but in 2019, no one would believe that was even possible. As soon as the technology started to get good, we never cloned anyone without their permission.

>> People being confused with deep fakes. They still do not understand the difference between hyperelia and deep fake. is that one is happening with consent and the other is happening without consent. >> There was a place where it was my face and my voice. Suddenly it changes into three different tonalities of voices but still my face depending upon the kind of use case we talking about and then it switches into a female voice. >> These videos are air generated and these videos are real >> so that the audience can just be more

careful when they're watching air generated videos. I actually recorded I think like over 15 20 videos of myself and the model would never work on me right like it was horrible. Just the technology in our hands wasn't enough but a significant number of pixels in the world in the future are going to be generated. >> What are your some of your predictions around generative AI the capabilities? >> Making predictions in gen space is very very hard. I think everybody is wrong about predictions. >> What made rephrase stand out as against

other potential companies in this space to be acquired? It >> it'll be too sensitive. [Music] This episode is brought to you by Super Checkout, the world's first KI powered checkout that lets you split, combine, or co-ay with friends and family in real time, whether it's shopping, gifting, or travel bookings. Now, everyone can chip in with ease. Check out smarter with super checkout. The Hustle Group Company. Let me introduce you to the Hustle Group Company. It's a lifestyle brand that's redefining streetear

through the power of fashion, self-belief, and resilience. The second one being Discover Dollar, which is an AIdriven tech company that helps brands and retailers recover hidden dollars from overp payments and leakages. Docs now intelligent platform empowers businesses of all sizes to rapidly collect, manage, govern, and collaborate on the data front, transforming your documents and making sure there's an impact on the business bottom line, all in a secure and a single environment. >> Hi everyone, welcome to another episode of the Innovators and Disruptors

podcast. I'm your host Ab Tandan and today I have a dear friend Ashra Malotra who is here joining us for this exciting conversation that we're going to have around rephrase hyper reali and more importantly and most importantly Ashra's journey into building something so disruptive that the world was taken by a storm and specifically a lot of work that we have seen in India and outside as well now that's happening in this space he's going to give us or shed some light on some of these very interesting conversations welcome aboard uh and on the episode. Ash, thank you so much.

>> Thank you. I'm very excited for this. >> Thank you so much, Ash. You know, I I was just researching about you like when we started speaking back in 2020. Uh you know, you spoke to me about certain vision that you had about Rephase. This is before Rephrase became fully mainstream, right? You were doing some early experiments back in the day and uh but but when I was looking into your background, I realized that you are from a from a place called as Narura. Is that correct? Am I pronouncing it right? >> Yeah. >> Okay. So what's your journey been like?

Before I get into any stuff around rephrase, I want to talk to you about what's your journey been like you know you know coming from a place like Naura to IIT Bombay and I've seen you you know you've been fes 30 under 30 you've uh you know been at MIT media lab so what's the journey been like you know what got you uh hooked to the vision of rephrase eventually but before that what what key milestones happened in your journey? Sure. Uh I g you as rightly mentioned I grew up in a place called as Narura. It's a small village oblique town uh close to Delhi like say 4 and a half 5

hours away from Delhi. >> Mhm. >> Um both my parents are nuclear engineers. Uh so Nurora has a nuclear power plant where both my parents used to work. >> Um it was a fairly small community of say a couple of thousand people at max uh there. >> I had a lot of fun growing up. Uh I was not the most studious person I would say. Uh my parents used to make sure that like I have my basics right but like I was never in the top three even in that uh even in my class of like 50

even in that place. Um then in when I was in around ninth or 10th class um my parents you know all thanks to them they took a big sacrifice they resigned from the job. uh they got they then like you know one of my parents got transferred to Bombay um and that's how I moved to Bombay. >> Okay. >> Um my parents didn't want to send me to Kota for uh education. So like they want to make sure that I'm at least in a more competitive place within like close to house. Um Bombay was a big culture shift for me. Um going from a very very

protected like you know very small community to like now this place where you almost have to like people are more like out there, people are more modern, right? like uh >> and there a lot of people. >> Yeah. Yeah. >> Just a lot lot lot poor people on the street. >> Exactly. Exactly. So it was a big culture shift uh for me. Uh but uh I don't think without that shift I would have been where I've you know I wouldn't have never even entered ID Bombay for context. Um

I I did work really really hard in 10th 11th 12th I would say like those are the three years that I spent a really like a large amount of time. um my my parents uh did a lot of sacrifice for me to be able to get to IT Bombay and uh I wouldn't even say it was really my vision like uh I I wasn't particularly passionate about anything uh but uh you know when you have engineering parents and you don't particularly have a strong direction like that direction gets imposed on you which is go to IT right get a degree then go to IM then get an MBA and then get a job right like that's

a quintessential Indian path >> uh so I started off on that direction went to Bombay. Uh right used to run the film club in the second year. Um got very passionate about the world of world of like films and images and pixels like the fact that a bunch of numbers on screen can uh manipulate real human emotions is something that I was I got very fascinated about. >> Fantastic. Then uh in the third year like again on that eventual MBA path uh went to one of the largest investment banks in the world. Um I absolutely hated it. Uh and

after that I decided I should do more tech in life uh rather than going the MBA MBA direction. uh uh in third or fourth year uh like winters there was a two-eek program where uh professor Romesh Rhasar from Camera Culture Labs uh which is a part of MIT media labs uh they came down u there was a like somewhere between a full-blown project and a hackathon where they put a small bunch of people so in our team there was like one dentist say two or three computer vision engineers one business grad So they put together a crosscultural team

>> and they gave us a goal of >> um can we build a camera within a toothbrush >> uh where you can get a report of are there any parts of your mouth where you should be focusing more on or less on. >> Basically it's to avoid cavities rather than treating them. >> Again computer vision >> it was yeah it driven by computer vision. So the signals that are coming out of the camera, you've got to uh run some algorithms on them. >> And of course after a week, that thing did not work reliably on all possible

edge cases. U but it did work in at least artificially created setup. >> That was a huge mental change for me. >> Okay. >> Because before that my belief was that you needed teams of hundreds of people, you needed millions of dollars to be able to build out any product, right? But for the first time I saw that a team of say 5 to 10 people uh across disciplines in a fairly small duration we can ship out uh something that looks like a pro type or a product right >> so that was a huge mental shift for me >> uh which uh then you know that continued

into a full 2 to three month internship that I did remotely like to the red x program that was based out of India that camera culture group from MIT used to run Um so in the summers I was doing three full-time jobs at the same time practically I was doing Google summer of code I was doing my master's degree and I was doing uh like this Redex program at the same time. So I got used to working really really hard just because like you know you have to paralyze three things in at the same time. um in the final year of college uh two out of

these three things of course like eventually like my Google summer of code was done my MIT made a lab internship and then I had too much time right like when you get used to working those hours like I had so much energy and so much time left >> so before I pass out of college I wanted to build something that's just extremely memorable like like basically do something really cool before I pass out of college that cool thing turned into my first startup eventually and then like that first startup turned It's the second startup and I think that's how

the journey happened. >> Wow. That that's very inspiring. Very interesting question. Did you said that you did not want to go to Kota? >> My parents did not want to send me to Kota. I don't think I had like I had the cognitive ability to process whether I wanted to go to Kota or not. I had no idea what it was then I was in sixth or seventh. So for the audience there's one thing that there's a takeaway over here which is you don't need to be in Kota because I I think a lot of youngsters should have a lot of pressure to go to Kota to prepare for cracking IITs right

they they you don't need to necessarily be in Kota only to prepare for a good B good engineing school >> uh that's true uh my my takeaway from spending a bunch of time with with people is that most people in in IIT in my opinion fall into two categories First category is people who are like just >> crazy smart, right? Like they're just born smart. You do anything, you put them in any situation, they will be the smartest person in the room. >> You're one of them. >> I don't think I'm one of them. All

right. But like there is a category of those people and like they exist. I think the second category of people are people who've had to work really hard to be able to get to that point. Mhm. >> Um and I think when at least I was younger, right, like working hard can either be imposed on you by some other like you know someone else. So your parents could like force you to do it, your teachers could force you to do it, right? You could put an atmosphere like where that's happening. >> Or the second kind of place is where you yourself feel bad, right? Like even as a

kid you have a little bit of selfrespect, right? >> And you don't want to come like say 15,000 in the country like you feel bad about it yourself. So uh uh what is what is really uh helpful and I think is important is to be put in a group of people where you have a lot of competition so that you have like you know self motivation to work harder and selfm motivation to like improve your ranks improve your marks and and just like generally get better at whatever you're doing. So Mumbai gave me that opportunity to like have peers who were

smarter than me and that makes you like naturally want to like you know get inspired. >> Um so I think that is very important. Now kota is one way of getting there which is that you see a lot of people who are smarter than you than you. But I think if you can replicate that situation where again just don't be the smartest person in the room. Make sure there are other people better so that you you have you are inspired to catch up to them. >> Right? I mean I used to always get this uh statement uh by my dad right uh back

in the day much younger you know when I used to come maybe in the top five in the class or something like that and he used to tell me it's like a kua commander right you think that you know you are very sharp but then when you're out there in the real world uh in a real competitive environment that's when you really understand uh what it takes to really crack something right to really build on something and that's where you'll strive a lot harder and I realized this in my story I realized that when I was put at uh SPJ and cracking CAT was comparatively much

easier you know one of those competitive exams but once I was inside SPJ is when I realized that you know everyone out there was probably sharper than I was and I'm like wow this is this is interesting because I was not used to being in a setting where the competition was so high and then I was like okay now I have to pull up my socks because if I have to compete in this environment you have to be you know learning from each other at the same time there's also that u extra energy where you strive to get beyond a particular point Right. So the escape velocity so to say right to go to

the next level pretty interesting. Thank you so much for sharing that. I've also been a big fan of uh MIT uh media lab and the work that they do right in in the past some of the research that we have done around nanotech also has been inspired by some of the stuff that we've already seen that has happened at MIT even at target we had partnered with MIT media lab. So I know the people who have come out of that normally I mean you you said that you know it was in India but uh I have seen even in India the people who are selected are normally quite sharp that's that's the point that you

were trying to make that you know there are only five or seven people in each group but such diverse crosscultural teams that were made that each of them was carrying not just their they're not you know bearing their bates on their shoulder but you know together as a team those five people turn out to be a 50 people strong team. >> Yeah that's true. Um I've as I've realized working in large teams now uh 50 people adds a lot of head lot of bureaucracy and process of just people management. Um so I don't think like my naive version say 10 years back you used

to think that 50 people would get 10x the work done of five people like that is just so not true. >> Yeah. No I I completely agree when I was uh leading Lowe's innovation labs in India uh we formed a small team of 8 to nine people. Each of us knew exactly what we were doing, what we wanted out of each other. We created a a term or role called as creative technologist because we wanted one or two people to not be bogged down by presentations, not be bogged down by a lot of administrative project management or product management work as well. Their

jobs was just to do creative experiments. They were technologists who were also very creative in their headspaces. So we got those people to come up with radical ideas that we could do disruptive work on. Right? In fact, in one of those conversations that uh I came across Rephrase and uh this was uh I probably uh saw what Rephrase is capable of uh at Samsung's Opera House in Bangalore and you were at the demo day of tech stars and I suddenly see on the stage you're talking and there's this Barack Obama audio clip that's playing out right and I'm like wow

what's going on right and then that's when I I think me or someone from the team reached out to you saying that you know let's connect and there should probably be lot of things. So what was that journey like? you know earlier conversation very very very fascinated by movies very fascinated by the fact that certain things which are denoted I mean movies are nothing but numbers data that is coming up on screen uh different visual representations of that and that's manipulating human emotions right uh and one of our first conversations you mentioned this to me

that your intent was always to come to a place or the long-term vision northstar if I can take the liberty of saying so to convert text into movies. You write any text and that should automatically convert into movies. What was the journey like from uh you know figuring out what you want to do to eventually getting to this place where Barack Obama's voice was cloned. >> It was a lot of fun. Uh so Nishe who was the co-founder uh along with Sham um at at Rephrase so it was Nishit's idea that he wanted to build uh even in college a machine that takes text as input and

creates a video as output because back in 2019 computers did not have the vision ability. Uh so you add all all the challenges of natural language processing to be able to truly understand what the what the text is and then replicate that with all of the hardest challenges in computer vision of being able to make uh the machine say that it all sounds pretty simple today but in 2019 no one would believe that was even possible. So uh so Nishid had that idea and then when he was discussing it with me um that resonated

very deeply with me because I used to run film club and like I used to make short films and I made them for like a couple of years but after that I got fed up given how complicated the video creation process is like it takes weeks months to create even like 3 to four minute short films you're thinking about it all the time you are at a set right like a bell will ring a crow will come down right like something something goes wrong all the time in in shoots and hence I was convinced that this is something that there has to be an easier way of doing it. Um I eventually end up

going to photography because that was just a lot simpler to do that than than video creation. So the idea that video creation should be easier >> right >> uh and if you look at it from an engineering first principle mindset and then Nishit came to me with with with with this vision and I think that resonated very deeply with me. So uh that was the uh starting point of rephrase. Mhm. >> Also we were convinced that uh we like within the team we had the skill sets to go to this problem go at this problem

and that there were this was a new space that was coming up. So with any other space we thought that you know we might have like we were still like fresh out of college grads or two three years experience. So we thought in any other space we might be missing a lot of experience other people might have but in this space it was like you know it was fair fair game. right? >> We we are in the same place that everybody else was. >> Um also we were seeing some offshoots in research like say there was a paper by University of Washington called a

sensorizing Obama and some of the research that was starting to come out. Uh so yeah I think like it was it was some early like saplings that were coming out. We just put all of them together and decided that okay let's this is the time let's make it happen. text video is like is going to happen and we thought like we could be one of the teams that will that will make it happen. >> So that's how we started. Uh we took uh Obama in the early days as the research because the earliest models needed a lot of training data to be able to train um

and uh Obama used to give a presidential speech every week u uh which was put in public domain by the white house. So and he gave that I think for four to five years. So >> he basically had say you know a few minutes of his video every every week for like a very long time. >> Uh so that added up to I think like I may be getting the number wrong but over 20 hours of data. >> Um which is what the earliest models needed to be able to create a uh a model that can synthesize face and voice of a person.

>> So that's how that's why we started with Obama as an example because public lots of data public domain. >> Amazing. Did you ever get into any kind of trouble because of the Obama video? >> Uh I I don't show that video uh as often now given the increased sensitivity of deep fakes and how easy it has become, >> right? >> Uh but uh no, I don't think we serious trouble because of that. M >> uh because Rephrase followed fairly strong ethical guidelines

>> outside of these few like fancy demos and even in the demo we never tried to make it feel as if Obama really said something that he didn't want to like we made Obama become Alexa >> right like it was it was it was supposed to be a cool demo we never wanted to spread misinformation and outside of a few demos like that which we did in our very very early days of our journey we almost like as soon as the technology start to get good, we never cloned anyone without their permission. >> Uh so no, I think we followed very strong ethical guidelines. None of our

campaigns went wrong and I know many of our competitors have had that. >> Uh so it was like following the strong SQL guidelines is important to us. >> We wanted to build a technology for good. >> No, thank you so much Asher that that's fantastic. Right. Two very interesting points that you spoke about. two very interesting points that you spoke about right one of them being uh the importance of following ethical practices. Uh second was people being confused with deep freaks. Now because I have closely worked with you over the

years and I have followed your journey as well I understand what hyper reali is what deep fake is but I'm sure a lot of people in the audience or actually outside in the world when I still talk about talk to people about hyper reali even though chart GPT all the rappers like perplexity are out there people talk about them they understand what generative is all about they still do not understand the difference between hyperia and deep fake right uh would you want to share a little bit a perspective so that people in the audience who are who are watching you understand what's

the key difference or what are some obvious difference between hyperi and uh defake >> yeah I think the largest difference that I think between both of them is that one is happening with consent and the other is happening without consent so there are lots of cases where people want to create uh video content of themselves >> without having to actually go out to shoot lots of use cases >> right It scales up people's time >> because now they could be somewhere else and you know a video is getting autogenerated. Um in the case of the

Shah Rukhan campaign we created or like many other celebrity campaigns we used to create hundreds of thousands of minutes of content and the person just does not have that much time right >> uh so in the Shahhan campaign just taking that one example we created more minutes of video content of him than his entire film life put together right in one month. Okay. I mean in terms of creation. >> Yeah. In terms of >> not for the training data. >> No. Yeah. Only in terms of creation. >> So the fact that we are able to create

more more minutes of video content in one month than he's done in his say 30 40 years of film life. >> Wow. >> Is is only possible uh with AI >> right? >> So that's to say that there are lots of use cases where you want to create AI generate video content of a particular person for good reasons for ethical reasons. Right. Right. All of that happens with their permission. >> Coming to the other side is deep fakes. Hm. >> Deep fake's origin was like swapping out

faces of like face swap. Actually, not even generating the entire video. The origin of deep fakes was face swap and it was face swap in porn videos like swapping out people's faces, >> right? >> For obvious reasons, that thing has a bad reputation, right? And it should. >> I don't think AI generating people's videos without their permission uh is is right. Hm. >> More so, I think using those videos and not letting the audiences know that these videos are air generated is worse. >> Right.

>> Uh I really wish that the world figures out a mechanism >> to say that you know >> these videos are air generated and these videos are real >> so that the audience can just be more careful when they're watching a generated videos. >> Makes a lot of sense. >> Um if that happens I still believe there will be a lot of use cases of a generated video because a lot of times you just don't want to read text or listen to audio. we want to see a video that's generated. It just feels more

interactive more than just reading text. >> So I think there are lots of use cases. >> It just needs to happen with consent and permission and being transparent with your audience that are generated. >> Makes a lot of sense, right? So when when Ashid speaks about uh face swapping, he's talking about superimposition of someone's face on someone else's body, right? And that like you mentioned uh the prime use case has been born uh and thus the reputation because of the stigma around that right but more importantly it is

permissionless which means that you know this can be used for bad reasons uh you know as the fake and uh something that most people I mean a lot of celebs have also gone through that in the last few years right uh so clarifying again that hyper real is different you know you're training the whole model on certain amount of training data then you can generate script which can allow that model to speak out things with permission of the actor in question. Right? So that's very very critical that you know people understand that difference.

>> Yeah. >> And that is where that is why it's very ethical also as against you know permissionless. Uh I remember ash uh 2020 mid just after the onset of the pandemic uh we engaged uh I was I was working at Lowe's innovation labs and we engaged to do a you know proof of concept or a pilot right and one of the first you know lab rabbits uh in that experiment was me and I I thoroughly enjoyed it no matter how difficult it seemed at that point of primer facy because I remember you know I I'd called

you up and I told you that you know I I don't I can't have people coming into my house. We are one of those people who are very very uh skeptical about contact at this point of time. So can you let me know if you can set up stuff at my house only instead of coming to a studio environment and you said sure we can get the bags you know shipped to you and then you can you know give you instructions over a call and you can set it up. And I said okay that that's cool. The funny thing is the bell rang and the guys kept bringing the bags after bags and I'm like how many bags are there?

And then I realized there were almost 10 bags and it took me almost three to four hours to set set it up. Now again coming from a nonmovie background not non-production background even a simple thing I don't even know what was the term being used but it was a horizontal versus vertical for the green um green screen to be put up right in a particular format and I was like I'm not sure what you're talking to me about so there were video calls there were a lot of audio calls and then I was trying to figure out the light settings camera and all that and I think for that shoot

there was no teleprompter of course and uh because it was a early model you had asked me to keep speaking for 30 minutes or so without a teleprompter where I was the cameraman, I was the crew, I was the producer, I was the director and I was the actor. It was really fun. Uh but but that was not the sacrifice, right? The sacrifice for me at that point of time was since the algo was fairly new. You had asked me for a big sacrifice. I'm not sure if you remember. >> I I thank you for going through all of this. That none of that sounds easy. >> Yeah. No, none of that was easy. But the

biggest sacrifice was that I had to shave my beard off. And the reason it was a big sacrifice was because I look like a egg without a beard. So for uh a few days I had to bear that look. So I had to take off my beard. Uh I had to take off my glasses and then I had to keep speaking. Now it becomes difficult for algorithms to understand what the lip movements are like at least back in the day because of heavy beard and mustaches right and eye movements with the glasses on. So that is why my assumption was that you know these were I mean I was made to do that. I hope so.

>> Uh yeah yeah uh uh >> there was a hidden agenda. >> Uh no there was no hidden agenda. That was just an unfortunate moment in time and the algorithms needed all that support. Uh as I mentioned like in the earliest days we used to train on like many hours of footage of Obama. Uh we then brought it down to something closer to 30 minutes that we saw. >> But again we're talking about 2020. So we we needed two things. First was again you needed to have lip like points on your lips etc. And like the landmark detector uh would fail if you had too

many like beards and stuff uh and and glasses. So like that was one problem. Right. >> The second major problem was that uh again we used to generate like this area. We would not generate the whole thing. So we generate this area and then merge it with the original video. >> Um and the generated part was not as high resolution as the original video was because we were not able to generate at such high. >> So uh there was a little bit of smoothing >> and the smoothing does not feel as bad

if you know like if you're if it's just skin because but if you have beard which is which has fairly high frequency details in this then the smoothing feels really bad. M >> so uh that those are some of the reasons why we needed to do all of that. Um but yeah uh I'm you know I think all thanks to you and and like all the support that we got in the early days. It was like we had to build up one case study at a time right like it started off with like you know people in the team like >> I actually recorded I think like over 15 20 videos of myself and the model would

never work on me right like it was horrible. uh like I was so it started with people in the team and then it started with like you know very early adopters like you and like we had to build up our library of content of people that we have cloned to be able to then go on and convince like the talent management companies to trust us initially with the campaign and then and then it all grew from there. So yeah it was those are all like very very very helpful steps on on the journey. So thank you for going through that. No, my pleasure. Complete my like I like I said

right I've been very inspired uh by this whole exercise that we did. In fact u I remember that brainstorming conversation that you know both of us had about presenting this to senior execs at Lowe's and we were talking about how do we give them a presentation that really markets the technology so damn well that they're like in awe of what is being done. So we had a lot of uh I mean you know played around with the video content right? So if you remember and I'll probably you know add this for the audience to see shortly in the video. Uh there was there was a place where it was

my face and my voice. Suddenly it changes into three different tonalities of voices but still my face uh depending upon the kind of use case we were talking about. And then it switches into a female voice. Hi, I am your virtual aay synthetically created. Till now creating professional quality videos has been a huge high production effort. But now you can just write the script and get a video within 5 minutes. Isn't this awesome? >> It's also super easy to change my voice. So I can have a casual voice with some customer

>> or change my tone when speaking with other customers. And remember this involves no voice casting, auditions, recording, nothing. >> And yes, I can also switch into a female voice, by the way. >> Mind-b blown, right? But this is just the start. There are a variety of faces to choose from and mix and match them with different voices each time. Let's say if you are shopping on the Lowe's windows section, I can show you around. And when it comes to plumbing, I can tailor my avatar to make you comfortable and confident in a buying decision. And

this is just the start. Good to be back completely with my face and my voice. Let's step up together in the future of video creation. And I remember uh some of these leaders laughing really laughing it out because they did not anticipate or see expect as it is the the egg face was funny for them because I was sitting right next to them with a beard and they were like this guy's looking so that's that's you. And I'm like yeah that's mean I'm beardless and they're like oh my god you look you look like this without a beard. So they were being nice to me without

explicitly calling it out. But yeah, and then the next thing is that they hear me with a female voice and they're like, "Wow, there's some possibilities that exist, right? The crazy amount of possibilities." And I I informed them that while we are just talking about this, we talking about different languages. We're talking about different background settings. We're talking about different musical scores. We're talking about endless number of scripts. And initially the pitch was about email marketing campaigns. But then when it went on to influencer marketing it

changed the direction like you mentioned you know Ryik Roshan was one of the first ones uh you know celebs that you worked with for again if I'm not wrong a Cadbury ad. Yeah, >> Raaka Bandhan. >> Raan. That's >> And it was smooth. It was damn smooth, Ash. I remember sending it to my cousins. Uh, and they were like, "How did you do you really know anything?" Like, "Yeah, of course." It was damn smooth, right? So, it kind of worked. That was a way of testing it in the in a you know unbiased uh audience. And then

u Shah Ruk Khan's campaign uh not just a Cadbury ad uh for Diwali changed everything. I think that was a big moment. Could you talk to us a little bit about what went into that campaign? Because if I'm not wrong, you got 1.5 ret 1.5 lakh retailers got Shah Rukh Khan as the brand ambassador. People who can't think about affording Shah Rukhan as a brand ambassador certainly had an option to have Shah Rukhan as a brand ambassador selling different kinds of products for a Diwali ad for Cadbury. some something that you would like to share about what what really transpired

what how did that scale happen? >> That ad was also a uh transformative moment for us as a company. So uh uh it was a important moment. We I think it all started off with the Rythic campaign that we talked about earlier. Um, Cadbury used the Ryiki campaign as a testing bed for whether any of this stuff even works or not >> and it worked really well. >> Uh, so they were very happy with the results. Um, Diwali is the main event for them and Aguili which was a creative agency that uh, Mand

had at that time. I'm not sure who's who's it today. um came up with the idea >> that in Diwali uh can their goal was to reach out to tier two tier three uh retailers >> and uh they came up with this idea of let's create a personalized shark video >> um and they could now take a bet on us uh because they had done a mock test earlier um so it was Augil's idea um and with Wavemaker who's a media agency and stuff like Mandlay is Agilv, Wavemaker, us right like and like another tech partner like so we all work together to first even do the feasibility test of

this whether this is possible or not >> right >> it was a very big moonshot at that time >> given m said that we wanted to look like a normal ad >> basically like even the kind of stuff that you had done >> it was a person standing looking at the camera >> and you know giving a 15 30 minute uh long speech. They said we don't want that. We want it to look like a natural ad. So they wanted to push us push the boundaries of technology more than it ever had been pushed before.

>> Makes sense. >> So as a result of that they said that we have five different categories. I think one was shoes, right? Eyewear, groceries and like I think a couple of others, >> right? >> Um and they said um we need five different like he needs to be able to do personalization at five different times, >> right? The model at that time was extremely sensitive to you know lighting changes and like even costume changes because we would like capture on like the model was extremely sensitive at that time. Now of course eventually the

technology got better. >> Yeah. >> So we we initially pushed back >> like that's too ambitious. >> Uh let's just do like one avatar one place. But they were insistent like you tell us what you need even if that sounds irrational and we'll figure out a way to do it but it needs to look real. Makes >> sense. Um, so we ended up creating five different avatars of Shah Ruk >> in five different like parts of the studio and different lighting, different

clothes, all of that. So Shahon actually had to speak like gibberish training content was 75 minutes because the initial 15 minutes for each training. Um, I think it was extremely patient to go through that. It was it was very hard to speak like random stuff for that long. Uh but he went through it. Um that campaign had a bumpy ride on the way to getting signed off. There was some like lots of challenges uh even in say like Shahuk's personal life and like you know the sensitivities around the situation uh at that time. So we weren't

even sure whether it would go live. Um and I think after it went live it changed the game for again both us as a company even for the Indian advertising scene. I think India won its first ever cons uh like major cons awards in the which are the equivalent of Oscars in advertising >> titanium lion. >> Yeah. Yeah. We won titanium lion. Um but more importantly for me um we got emails and messages from people who said that you know for the first time they believed in technology uh because lots of lots of these people

>> all they wanted to do was you know stand out of Shah Ruk's house and uh like get to see him once and in their life and that was their like their goal that they've planned for many many years >> but now the fact that you could be in any part of the country and um you could create a Shark Khan ad for their own brand. >> Um >> that's >> was like game changing for all these guys. So while we were thinking all this while purely from a technology perspective the impact that it had on

people and like how people perceived it was was very touching uh for us >> I'm sure >> uh and I think that's that motivated the entire team to do a lot more that motivated you know like us to push forward in our journey but uh again all thanks to like agil for coming up with the idea and then like wave make up for the media distribution mist for trusting us I till have a very very high respect for for Cadbury's and and model um I think they were one of the rare people who like took the leap to the bet >> because

it's very hard to trust a new technology a relatively new company >> for what is going to be the most important time in the entire year uh for your sales >> y >> u but they did trust us u and I'm I'm sure like their revenue growth was one of the fastest those in like four or five years as well. So I think everybody won that campaign. >> U we really enjoyed that. It's so beautiful Asha when you talk about this right because this is the exact thing that you know I wanted to get you for

this episode 4 because I wanted to talk about this impact there's there's been a big journey behind Nishid Shivam and you coming together right there's been a journey about your upbringing coming together as founders and trying to solve for problems right and then there was this dream team that came up you have mandbury partnering with KV wave maker and rephrase pushing technology to the best possible limit to come up with this and that's when disruption happens that's when a new space is carved out now

masses are aware about the possibilities that exist two it solves for you know it brings happiness and impact in the lives of those 1.5 lakh retailers of brands who are now able to say with with Shah Rukh Khan saying this out you know ma footwear from MG road hypothetically you know as example and people were able to say that brand and that was so beautiful right and you got so many emails that's a testimony to the impact that you've created and the fact that India got its first uh titanium lion at KHN's it shook the whole

advertising space because in all these decades of India trying to participate and win that award India had not and for the first time technology was embedded right in the center of something so creative right because people would write off technology saying that you know creativity is different technology is different and then you showed that you know it can coexist and not just coexist it can win together so it was such a beautiful story thank you so much Ash for sharing that >> thank you and yeah creativity and

technology go really really well together um as I mentioned like just the technology in our hands wasn't enough. We needed amazing creatives to come up with amazing ideas on like the impact that could have. So multi-disipline teams coming together for a common goal works really well. >> True. Very true. And today you see these technologies being deployed everywhere. I've seen uh celebs uh across cricket, sports, athletes, uh movies, etc. using this technology today, right? in different form factors and it's fantastic to see that you know in four

years we just talked about 3 to four years how much has it evolved in terms of adoption and of course it has had its own challenges across the journey as well along the journey as well I'm sure you've gone through some of them one of them that I didn't want to talk about was uh fundraising I'm sure this is not an easy space to sell your story and narrative to investors because this is not a very proven race. So how how were you able to go through that journey? I think you raised about 10.6 mil series A. One of the people that I respect as an investor

a lot is Himan Moapata Lightseed X A16Z if I'm not wrong. >> He was also on board uh Red Ventures and a bunch of others. Right. Can you talk to us a little bit about your journey about fundraising and how this narrative was sold to investors? What what made them believe in you? >> That's a great question. Um so Hmon led our seed round um Red Ventures Lake 8bc and a bunch of others invested in our series round. Um in the techstars demo day uh we originally went off as just you know like to show off what we built with no

goals to fund raise given we like I was coming off from a failed startup where like we basically died because we could not raise money. M >> so this time you know we'd gotten I think a little over 100k in India that was a lot of money for the three of us to survive so we're like we will not fund raise we'll just stay lean we'll build it out >> right >> but Hmon again all thanks to his visionary abilities he was able to like predict that this technology will will evolve into something uh a lot more

>> u >> and you know after talking to like the team and and his vision of where the space will go he decided to tried one of his first checks moving to after moving to India into us. So I have to like commend him back in 2019 it was a very very hard bet to take. So uh yeah I think that a lot of the round the fact that the round happened like credit goes to him. >> Ash can I just interject on this and ask you this follow on question about Hmon right since you've worked very closely with him. Do you and you've worked with

a lot of investors you've spoken to a lot of them I'm sure. uh do you foresee that because Hmon moved from US to India that his risk takingaking appetite on emerging technologies in unproven uh you know spaces was a lot higher than a bunch of Indian investors who who normally would like to see that you know has this technology been proven somewhere else and then they decide to invest into that. I don't have a large enough sample size to be able to generalize the fact that say people coming from US20 are more risk takingaking or not like so I can't

make a comment on that but Hmon in particular sure like he's gone on to write amazing checks you know into pixel and many other deep tech companies he's done really really well so uh like Hmon in particular has taken a lot of bets and I have very high respect for him um whether that statement generalizes to all people from US to India coming I'm not sure >> okay fair so you continue >> so then our series A round um happened because of a few different reasons. First, I think the success of the Shah Rakhan campaign showed that this

technology could go mainstream because before that like it was all very niche. Um secondly, we found a very fortunate partner in Dread Ventures where u they were pivoting like away from just pure play text based publishing >> to wanting to scale up the video content. >> Uh and there were lots of synergies between what they wanted to do and what a technology could enable. So that's what prompted the entire thing. Uh and we looked at them as very important strategic partners on the way to US go to market because I was convinced that

we would have to pivot our business away from just being India focused to having to focus significantly on US as well to be able to get to our series BC and and follow on round to be able to get to that revenue revenue scale and growth. So we wanted very strong partners in the US which is why uh I think red ventures and 8bc and silver lake were all amazing partners >> that that's that's fantastic right and since you spoke about mainstream u you know going I mean hyperi or this technology at large going mainstream I've noticed today that you know you

know you can just navigate that arena so quickly like for example your apple can be changed your look uh looks can be changed very very quickly right so you can be placed into any place any envir environment very very quickly. That's the personalization depth that you know we're talking about today, right? I mean that's fantastic. >> Yeah, we'll talk about that. But like what do you your appel looks pretty nice. What do you >> Thank you. Uh this is a champions t-shirt. You know we are we're going through the champions trophy and u one

of the brands that I support is called the hustle group company. They're into pop culture and athleisure. So support the Indian team. I support the Indian team too. And basically it says that you know they want to make people feel like champions. So yeah but thank thanks so much for noticing that this this is these are the years that we have won the you know international trophies, world cups uh India at large. So champions. >> Nice. >> Thank you. Uh getting back to your original question, um I believe most okay I should say most

but a significant number of pixels in the world in the future are going to be generated >> right >> uh and not actually recorded. Um, a big part of that is being able to get all sorts of customization abilities and like sophisticated controls on a piece of software. In case of real people, that does mean that you know you have to have almost infinite configurable options of like of lighting, clothing, u like in the future being able to like air generate characters of any description that you

would want. um all of that's going to happen and uh I'm almost frustrated that's taken us so long to get there but uh I think it'll now happen sooner than people expect. >> Yeah, with the LLM's growing so fast. So this is also another aspect that I wanted to get into. What are your some of your predictions around generative AI the capabilities in the next let's say two or three years and I know 2 three years now seems to be a very far off time. Let's say let's one next one year where do you see generative AI going? um

making predictions in gen space is very very hard >> right >> uh I think everybody's wrong about predictions um think of about think about how this space really works right >> u you put together a lot of compute a lot of training data into a model >> you then train the model >> and then you see what comes out of it >> right >> which is very different than how say development in the prega world would look like where you would start off with a problem statement and then then put a

special dedicated like tiny model or like uh like algorithmic solution to the particular problem that you've start to solve because his model is so general honestly no one knows what their capabilities will be >> so even when say like Sam Alman is training his next models >> he can't tell you what exactly that model will do he can tell you what he's trying >> but very often like he'll build that next model >> and then they'll play around with it and then they'll learn all of the abilities

where like excels and where it doesn't. >> Of course, you can try to steer the models a little bit to like, hey, do better coding, do better at maths and stuff like that and like do more reasoning tokens and and a little bit of that, but honestly, no one has any idea of how the genai world is going to shape up. >> Makes sense. >> Is there a wall? Is there not a wall? God knows, right? Like uh we did almost hit a wall in terms of like how much compute can we put on training um like say GPD3, GPD4 kind of models. Um but

well that the fact that you have to put exponential number of compute to get linear uh improvement in performance means at some point you have to stop right like sure like Elon Elon is now able to put an order of magnitude more than other folks but can you really scale up an order of magnitude even more than that >> right like can you continue to put 10 billion 100 billion trillion on that because the problem is not just training it's also inference inference also becomes much more expensive so then you need applications which need that

expensive use case so all of that to say pred predictions in the gen world are hard. >> The good thing though is that okay first let me be clear I don't see those innovations stopping anytime soon. Secondly, in a absolute hypothetical world, if they were to stop today, >> you still have so much to be built on the application layer, >> right? >> Which you can for which you can literally just leverage all the capability of the models that have been developed till date,

>> right? >> Like we still don't have amazing email assistants. We still don't have amazing like chat support. Like there is so much yet to be done in terms of automating programming. Um, right? Like programs like Devon etc. getting some headway with there but still a lot to be done. I still I still don't think we even like even the way most people write their eval frameworks and and prompts and stuff like can can become a lot more scientific and become much better than than where we are today. So we are like with all of and I think this is a rare

moment in history that way because typically you get very minor increments in technology and then like everybody in the application layer just like you know latches on to those moments to make sure that application layer and technology that's available is almost on par right and and that's uh it is a rare moment in history where your technology is so far ahead of the application layer >> that there is so much that we need to do here to catch up >> makes sense >> um and uh hence I am I don't get to if I was a if I was building in the space

today I would not fret too much about well hey how much are we going to go from here >> makes sense >> the third point though is I think there are many different layers of gen LLM what we've been talking about is a big part of it >> but there's also image generation models video generation models you'll have you know 3D generation models like the world labs and stuff coming out soon >> um >> I believe that the you know the chat GP moment for the image, video and 3D

models is still to happen. You could make a claim that in image it has kind of happened but no one's making billions of dollars in revenue on image models the way people are making on LM today. Um so I believe there will be a lot more growth on image and video. Um and just maybe given my personal interests in in the video space, I am really looking forward to to some of those those moments because these increments will allow us to get to the world of you know building like video creating videos where pixels are generated >> makes a lot of sense and I hope in the

near future you are able to contribute to some of those uh research as well which you know which gets us to that uh phase. you've you've done a bunch of uh very interesting shoots right with some very famous celebrities out in the country and uh what was your experience been like you know you mentioned that Shah Rakhan was very patient you know he did that training data speaking gibberish for a for for I think 75 minutes or 90 minutes you mentioned were there any interesting moments that you saw with with some of these celebs you know were there for moments or uh

you know some issues challenges that you faced at that point of time because there was also facing 3 4 that I remember a lot of celebs were now being very skeptical because of deep fakes coming in. So during those early phases and your journey did you see any interesting moments or funny moments? >> Uh I'm going to skip particular names and give you examples in abstract. Um there were there were like one or two celebs in the country who just ex who absolutely denied using this technology >> purely because they wanted to like not

be that accessible to their audience, >> right? >> Uh which was uh which we actually expected to happen at a much much larger scale. But we realized that most celebs wanted to really communicate with their with their audiences at a one-on-one and a personal way and be more accessible. But like there were a couple of those folks who who didn't. So like I think that was one Um, another major thing that happened with a with another big celebrity is almost all videos that we see from that person uh

have significant corrections to their face and and hairline and and stuff like that. >> Okay. >> Uh, understandably because like they've been in the industry for too long. M >> um and the way they typically do it is they're they're like you know VFX houses dedicated to particular actors where they would correct every single frame of that particular video. >> Oh wow. >> U the problem that we had is that our inference data would be the same as our training data. So we said if you want us

to do that you'll have to correct 90 minutes right like said 75 minutes in our case >> which was extremely expensive and like very time and we had no time to do that. So uh uh like if you closely look at some of our ads like you'll see like you know a switch from VFX to nonvx to like back to VFX again. Uh another thing that we realized was uh something that we had not planned for before is that our algorithm used to work on real world uh video shoots. >> But we realized that when you significantly do like VFX to the face um

it loses some of the harshness or like the real world artifacts >> uh which originally caused our algorithms to fail. >> Okay. which we had not because we never did like such significant modifications of the face before. Um we had never planned for that but uh and this had happened uh in a case where like we had promised to the customer that will work because our technology worked so many times before uh we do the shoot now we on timeline to deliver uh and now we realize that like our algorithms are failing just because

that reason so first it took us like took our AI team and kudos to all of those guys they worked really hard to first identify the problem and then ship a complete modification to our pipeline to be able to fix that problem. M >> uh but yeah lots of learnings during our our journey. >> Didn't you go go to some of these people or did the agencies not go to tell them or probably it's too difficult to tell that to a celebrity saying that [Laughter] I mean this is what it is man. Uh I I understand it's not possible and I by

the way I have encountered couple of those celebrities too big names uh you know who would just say no to a shoot because we wanted to do it with one of them too and they were like no no no no anything where you know there's a one-on-one messaging with my face going to a huge community I'm ready to do an ad that can be screened anywhere you can use those uh collaterals and stuff but not one-on-one personalized messages on WhatsApp to hundreds of thousands of people that that I have experienced too. So interesting. I I don't understand what's the apprehension. I think that

that makes it such so uh such a it creates such a strong impact, right? Imagine getting a message from Mahindra Singh Dhoni saying that uh Jessi RTR Black was fantastic, right? So on and so forth some examples like that. That was so cool. you know when you receive that you feel so good about it and you are more attached to that brand ambassador as well as the brand right so the loyalty increases and that is what you're seeking you know in terms of enhanced customer experiences >> exactly um I believe it's similar to uh I remember way long like a long time

back celebrities did not want to appear on television uh they only wanted to appear on uh because they were exclusive and they would only appear in big theaters >> because like you know like you have to come pay a lot of money to be able to see us like we are exclusive. We don't like you know appear on TV. >> True. >> Uh which is for other people. >> Um so that that took a while and then something similar happened in OTT where like you know it was not considered the real thing.

>> Uh like the real movies are still screened in theaters. >> True. >> Uh so then that transition happened. So I think there are early adopters and late adopters in every transition and and and that's what we saw. >> Amazing. On that note, Ash, I wanted to ask you this as well. You've been part of some very interesting startup accelerator programs as well. Techars, Alchemist, right? Huge respect to both of them. Ravi Balani is a big big advocate of very interesting emerging technologies too. Uh by the way, Ravi is

the one of the I he's the founder of Alchemist as well. >> The founder of Alchemist. So what's your experience been uh in in these programs and if you can also share some feedback recommendations uh or learnings from applying to and getting into these mainst huge uh you know accelerated programs uh to fellow entrepreneurs young entrepreneurs who would like to now you know kind of take up similar paths. First, when I graduated out of college, I was just an engineer who had no idea of how the real world works.

>> Um, and some of these accelators really really helped me build up some of that business muscle, >> right? >> Uh, you know, how do you do sales, how do you do fundraising, um, and like our our PMF journey, etc. would not have been possible without them. So, I'm very thankful to a lot of those. Having said that, uh I was looking back after like after my startup journey um and I realized I think people tend to over complicate this too much and put the startup journey into a box. Hm.

>> Um, if you take like if you look at it from a 10,000 ft view, all you have to do is build products that solve someone's problems that they will pay for. >> That is it. >> Makes sense. >> Uh like you and that's it. Like that's that there's just nothing else to it. Now in the journey to that, sometimes you might require business help, sometimes you might not. Sometimes it might require to raise money, many other times you might not. >> Right?

>> Um so but that is what say you know building your own company's journey should be it's about okay here's this problem in the world I'll go out to solve >> um because I believe you know I have a unique insight into something and that's it. People typically think of it in terms of well hey you know I am in this leg of this long journey like you know I need to go to an accelerator then I need to get raise my C round then I need to raise my series A round then then my B round and

>> like sure all of that might be an artifact of it uh in some situations in many other situations you could just avoid this whole journey >> right >> um but yeah like u yeah the 2015 16 version of me used to think that you know this the way companies the built is by doing this step >> but now like I think that the way companies are built is by just solving someone's problems and that is like people should never lose sight of it >> makes a lot of sense thank you so much

that was a very candid answer and really appreciate that thank thanks so much Ash thank you so much right like I said very very candid perspectives and loved the conversation with you on that note I do want to give you a hamper uh we've got something highly personalized for you Uh so this is by this for you. Thank you so much. >> Thank you so much >> and hope you like it. By the way, this is uh by a company called Escape Design. Uh they are one of our gifting partners uh and one of our sponsors and uh they do a lot of personalized gifting stuff.

So thank you so much again for joining. It was a fantastic conversation. Very candid, very crisp and I think our viewers will get to learn a lot more about the emerging technologies out there specifically about video creation and hyper reali. So love the conversation. Thank you. >> Thank you. Always a pleasure talking to you. [Music] Feel free to share your perspectives through comments. Subscribe to the channel and do send us some suggestions as well.