path-tracing-image

42 Students Tackle Path Tracing and PhD Papers to Put on GitHub

42 Students Tackle Path Tracing and PhD Papers to Put on GitHub

Whether you’re a gamer, a techie or just a regular news consumer, you’ve probably seen or heard the term “ray tracing” or “path tracing” in the past few weeks and if you’re still unclear as to what this “next big graphical leap” in video game technology is, 42 students, Phil McLaughlin, Louis Young and Eric Lopez can tell you (and show you) what you need to know.

42 student Phil McLaughlin began working on “Clive,” 42’s very own path tracing student project, back in October. The name came about when a friend suggested it on social media, however, it eventually became its own, very fitting, acronym: C.L.I.V.E., which stands for Computed Lighting in Virtual Environments.

After pitching the solo project (as someone who needed a team) and getting approved, 42 students Louis Young and Eric Lopez, who heard about it through word of mouth, joined Phil in February. The three have been making strides in this new 3D rendering technique since.

Path tracing: ‘Ray tracing’s big brother’

In a recent article, Polygon.com described ray tracing as “the ability to render and illuminate a scene in real time with cinematic quality.” This 3D rendering technique has long been considered “the holy grail of graphics rendering” and while video game developers have waited a long time to see it, the wait seems to have shortened significantly just last month.

During the 2018 Game Developers Conference (GDC), the week of March 19-23, 2018, Nvidia announced real-time ray tracing, dubbing it Nvidia RTX as a a feature of its next generation of graphic cards. This rendering technique has been “a dream of the graphics visualization industry for years,” said Nvidia Senior Vice President, Tony Tamasi. Following Nvidia’s announcement, top competitors AMD and Electronic Arts have followed suit.

While leading graphics companies are just now making their big announcements, one of 42’s student project teams, has been hard at work for months making advances to what they call “ray tracing’s big brother:” path tracing.

We sat down with the C.L.I.V.E team to learn more…

Tell me about the project and why ray tracing has been in the news lately:

Phil: “What we’re doing is what’s called path tracing. Ray tracing is sort of a common graphics thing—it’s how a lot of CGI movies are made. Path tracing is kind of ray tracing’s big brother. Ray tracing has been sort of a holy grail in graphics for a long time but it’s always been decades away to have it in real-time or like interactive kinda thing for video games, basically and then all of a sudden companies like Nvidia, Microsoft, Electronic Arts (EA), AMD, everyone is making these huge announcements about real-time raytracing is coming now…like the next generation of GPU’s is all gonna have hardware support for it and Microsoft is putting into DirectX which is like the core graphics pipeline that all games go through. They’re experimental features for now and there’s no games that use it yet but all of a sudden it’s gone from being distant future to being like next year.”

How would you explain the difference between ray tracing and path tracing?

Louis: “The thing about ray tracing is that it’s actually pretty simple and intuitive when you really think about it. At its core, what it is is simulating light so you actually take literal rays of light — lines — and you cast them through a scene that’s basically constructed of a bunch of triangles and when the light hits the surface you use something called a BRDF to calculate exactly the direction the light would bounce in and that is kind of probabilistic so the light doesn’t always bounce in the same direction. It’s a little bit random but in reality that’s not really the case, it’s just that there’s so much light that you can never really simulate all of it so we do our best to simulate all of it by randomizing the way the light bounces off and by doing that enough times you reach a sample size. […] The more rays of light we shoot, the more accurate the image appears over time and it obviously takes more time and computation to reach the more accurate images so a lot of what we do is revolved around trying to make that fast so that we can cast as many rays of light as possible and get the most accurate image.”

Phil: “Ray tracing and path tracing get kind of talked about under the same umbrella so the distinction between them can be quite blurry. The big contrast in the process that Louie was just describing is ray tracing sort of has knowledge of the scene in advance so it tries to steer the rays towards the light and tries to explicitly point the rays in directions that will clarify the shadows and stuff like that which makes it faster but also—randomization and using statistics to approach something is very fragile and if you try to bias it in any way, it breaks—so path tracing has to not have any pre-knowledge of the scene. It’s the paradigm of many dumb rays versus few smart rays is the contrast. The rays in path tracing just follow the probability; they are not headed toward the light source or anything they either get there or they don’t and you kind of just let the math take control.”

How did you become interested in this type of 3D rendering?

Phil: “I knew about ray tracing from reading about it but I wasn’t particularly interested in graphics prior to coming to 42 but this has been something really interesting for me. What I knew I was interested in was like scientific simulation and I worked on a physics simulator project before this that was really cool but one of the things that was frustrating about it was how hard it was to see the results: it was just points and lots of big numbers and even when we did make a graphic it was just like very bare bones. Anyway, this is also a scientific simulation, also high performance all that kind of stuff but there’s raw visual impact and the thing that you get out at the end is much more rewarding. You can also just have a sort of intuitive sense of whether you’re doing it right or not. You can look at ten million stars orbiting each other and you’re never gonna know with your eyes whether it got closer to right but this you can see when it gets cleaner, you can see when it gets better.”

Louis: “They do offer a graphics branch here and they do have ray tracing projects – specifically two ray tracing projects – but they’re not exactly the same thing because they are more framed around real environments but they are at the end of the day, training you to cast rays in a virtual environment so they do offer the basic level of it.”

Phil: “This actually started as me doing the first of the ray tracing projects and getting distracted by how deep the field is. I still haven’t actually turned that project in.” (Laughs) “Been working on it for six months now, I guess you can say.”

What are your hopes for the project?

Eric: “I think the hope is to provide a powerful training tool so that just about anybody can come into our landing page and be able to take in all the resources that we provide for them. We’re gonna do a lot of the dirty work; we’re gonna try to dumb it down for them this way it doesn’t have to be a PhD student, you can just be a regular student or whoever you are, you can come in, learn what we’ve provided for you and become and incredible path tracing artist.”

Louis: “I think there are a lot of people that desire to get into this rendering because it is exciting and it is like simulation but there’s a lot of barriers because of what it takes to get involved with it. There’s a lot of very high profile papers and articles on the Internet that you can find but they are very hard to sift through, there’s just so much lingo there and it’s just very overcomplicated and it’s not consistent with all the different sources like they all kind of name things differently. Generally, there’s no one stop shop for path tracing…for ray tracing there is but path tracing is a little bit more obtuse. So we just want to be that and make it good.”

Phil:  “There’s just this really big divide with path tracing with how hard the math is to prove that it works and how hard the math is to just do it and so all the papers are just really rough. It’s very hard to cross reference because of the terminology—everything’s kind of muddy and people use different variables. What it takes to understand to literally just make code that does it is much more accesible than if you try to enter through reading papers and stuff like that and currently that’s the only path available so we wanna make something that’s more a learning tool— we’re adding lots of visualization tools and ways to help people develop intuition about the stuff that was hard to develop intuition on.

There’s a lot of stuff that I wrote in the original code base that I realize how much easier it would be to write now with the tools that we have built instead of kind of doing it blind until it works.”

What other challenges have you come across?

Louis: “One big challenge was definitely the fact that Phil had started on this and had a lot of background by the time that both me and Eric had jumped on to it. I hadn’t done anything ray tracing related at that point and I kind of scrambled in the first two weeks to develop, like he said, this intuition. He had already done a lot of the work like parsing those articles and kind of mentoring me through that process so it made it lot quicker but definitely catching up and getting to that same point so that we can actually all contribute to the project was a big barrier. The first two weeks were basically just us building our own basic version of a path tracer from the ground up so that we could learn all the right knobs to tweak and actually make it work properly ourselves because if we hadn’t done any of that, then we probably wouldn’t be able to touch any of the numbers in the real code without screwing things up. Developing that real foundation—that was probably the biggest challenge.”

Eric: “For me personally, I would say that, because we do have ray tracing projects in our school which is like the the little brother of path tracingI had completed those two—so going from ray tracing to path tracing, that was the most challenging because it was definitely a paradigm shift in the way you have to think about it. Whereas, as Phil said, ray tracing is few smart rays and path tracing is a lot of dumb rays but with path tracing you get a lot more accurate representations of real-world light environments so for me, it was hard to shift and understand how it works and what it all means.”

Phil: “Path tracing might be easier to understand not having gone deep on regular ray tracing first because how you have to think about it changes completely. The way you think about what your doing when you’re path tracing is definitely a paradigm shift but in terms of the literal code path and what steps the code follows to achieve its result are largely the same—it’s sort of what you do with the results at the end but it’s always very counterintuitive.

I read stuff half dozen times proving that this works and I did not believe it actually did until I had finally written code that did it because I thought it sounded like non-sense. I kind of wanted them to go through that same path.”

‘For the love of the game’

In essence, what is the purpose of the project?

Phil: “We wanna make a learning tool or any intermediate step to help bridge the gap for beginners who want to understand more advanced concepts and that is definitely true and that is a purpose of the project but also, at the end of the day, it’s a lot of proving what 42 students are capable of and frankly just making really, really pretty pictures—that is actually a big part of the motivation goal.

I don’t know what the programming equivalent is but in sports people say ‘For the love of the game,’ like that’s kind of this for us right now. Just because it’s sick and we wanna see if we can do it.”

What do you like most about being here at 42 and being able to work on real-world projects?

Louis: “I just really appreciate that I’m able to follow a distinct passion and specifically for Rabbit Cloud, they allow you to expand upon that. They do give you a lot to work with in terms of the branches of the 42 projects and I am grateful for that, but allowing us to kind of pick our own project and if it’s something that’s really worthwhile they give us three months, which is a lot of time, to just really grind away at it. It’s pretty much an incubator where we have free dorming and just there’s a lot of opportunity to explore your own interests and I just don’t think that other institutions really do that in the same way. The best way you can explore your interests in college is just take a few courses off your major a little but you really can’t go that far because you need to get your credits and stay on course and here it’s nothing like that; it’s what do you want to do and then go out and do it basically and that’s awesome. Very empowering.”

Phil: “One thing that I just really liked about 42 is being surrounded by more like-minded people than I’m used to. I think 42 kind of attracts people who are a bit weird or have a sort of different approach to things and I definitely feel that I am one of those people and usually I am the only one around but here, we’re all like that and that’s really great.”

Eric: “I completely agree with what they said and also, I definitely appreciate we do get these three month periods to take on any project we want. In our case, we are working on this project that is usually done at a PhD level and 42 isn’t a regular college but we still get to take on these projects that other schools take on at higher levels.

Louis: “To get to the interesting subjects in college you have to go through three years of complete B.S. to get there and take all your gen ed classes and here it’s like you wanna do this level 4 project, go for it!

Phil: “No one here says ‘no, you’re not ready for that yet.’”

What sets this project apart from others?

Louis: “What sets us apart is we’re very technical and mathematical and low-level. We also don’t have a business aspect so we don’t talk about customers, but we do have users. We’re not selling anything.”

Eric: “But we definitely want to give people the opportunity to jump on this growing industry. There are these big companies that are saying ‘next year we’re gonna have these hardwares that are can allow you to do these kind of project easily so I feel like right now because they are doing this, the timing is perfect to give people the opportunity to learn and get in this industry as soon as possible.”

Phil: “It’s actually pretty interesting that over the course of doing this project it has gone from people asking ‘why would you be working on that?’ to ‘Oh yeah, I just heard about that!’”

How do you think this announcement will impact the industry?

Phil: “There’s a number of companies out there already that are using path tracing in film. So like most of the CGI that you’ve seen in movies lately was path traced and all of Pixar’s movies since Finding Dory have all been path traced. But those are relatively smaller companies and the people who are working on that ray tracing stuff are all PhDs and it’s just very difficult to get into. What I’m getting at is that there’s no entry level path tracing but now there will be. Now there’s all these other companies who just need to hire people who know how it works to explain it even. It was a narrow, difficult-to-get-into industry and now it has significantly broadened and it’s hopefully much easier to get into.”

The future of C.L.I.V.E.

The three currently dedicate a little over 40 hours a week to C.L.I.V.E. and while they all come from quite different backgrounds, they all agree that traditional college was not for them and opted for a more non-traditional approach as is 42’s.

The hope is that, now that these big companies are jumping on ray tracing, they will have the opportunity to land jobs in this growing industry. On his end, Phil plans to continue to work on C.L.I.V.E. as a hobby even after landing a full-time job and would want to make sure that it continues to be in some regard.

Phil McLaughlin: http://www.linkedin.com/in/philmcl

Louis Young: https://www.linkedin.com/in/louis-young/

Eric Lopez: http://www.linkedin.com/in/ericlopez28

Humble beginnings. Early experiments with glass. CLIVE’s Cornell Box render. Finally a complex scene! A big milestone for the project. Another shot of the same scene. Recently the focus has been on glass, especially colored glass. These are some of the current best. AWESOME!

published by Jennifer Robertson – April 19, 2018