What’s the tech stack behind the PyBites Code Challenges platform?

Transcript for episode 83 of the Test & Code Podcast

This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.

Bob Belderbos and Julian Sequira started Pie Bites a few years ago. They started doing code challenges along with people around the world and writing about it. Then came the Code Challenges platform where you can do code challenges in the browser and have your answers checked by pytest test. That’s so cool, but how does it all work? Bob joins me today to go behind the scenes and share the tech stack running Pie Bites Code Challenges platform. Thank you to PyCharm for sponsoring this episode. Check them out at testandcode.com PyCharm and boost your productivity today.

Welcome to Testing Code.

The podcast about software development, software testing and pipeline.

Today on Testing Code, you have the joy of having Bob Belderbos for people that don’t know who you are. Can you introduce yourself?

Yeah. My name is Bob Belderbos. I’m a self taught programmer since 2009 when I started building websites on the site in PHP. We had the awesome lampstack back then, I joined sun and started working in automation, so it was a lot of shell scripts. Then I got into Pro and that kind of got messy. And then around 2012, I discovered Python and I never looked back because it was so elegant and a great language to work in could refactor the code and have a long and happy life. And then I made a transition from the support organization where I was to a professional software developer role in the cloud in Oracle. Still have been there the last two years in charge of payment management product and now specifically last six months for the services side. So a lot of Flask and Django API stuff. So that’s very interesting. But you and me probably know each other better from Pi Bytes, which is a blog we started back in 2016 and morphed into a code challenge platform. And yeah, that’s the stuff we do on the side and it’s attracting a lot of people. We have a growing community of passionate Pythonistas.

If my math is correct, then you started Pie Bites before you transition to a full time developer.

Yes, it was around the same time it was December 2016. And professionally when I got the software developer role, I was doing software development kind of a oneman shop in the sport organization. But when I really transitioned into the team, that was April 2017. So that was shortly after.

Okay. It wasn’t something that your blogging could have affected you’re getting the job or not? Or do you think it did?

Definitely.

When you start blogging about typing, you do learn a lot more. So I did notice that when I was interviewing that that helps. Yes.

It’s one of those if you really want to learn something, teach it. Is that what you’re saying?

I can subscribe to that, yes.

Well, the code Pie Bites, I was excited to see you guys doing the original challenges and writing about it, and that was fun. When you started doing the code challenges platform. That’s kind of really fascinating. I’m guessing that not everybody has played with that also. So how would you describe the code challenges platform?

Yes, hosted Cochallenge Es, and we have something with Es domain names because I’m based in Spain and Julian is in Australia and with Au, you kind of get some cool names. So we have Pipes is the block and Coachella Es is the platform. And as you said, it all started with us doing life. No, not live like block code challenges for the community.

And then we wanted to have more like an automated way so we could people have submitted their code and run tests in the browser. So we started looking around and actually, the front end Python clients were not that complete.

And I guess we go a bit in the stack later on. But the Code Analyst platform just brief summary is an online coding platform where you can code in a browser. There are 208 exercises. Now, the tests are pre written, so you’re presented with a code editor, an exercise description, and a set of test and code. The exercise in the comfort of your browser. You submit it and the tests are run against your code and you get a pass or fill. And then you can accumulate scores and belts and their gamification.

Yes. Has come a long way.

And it’s pretty fun. I’ve been doing it off and on. I don’t think I started right away when you did it, but I’ve got a team of people, so we have maybe 15 people doing it together, and maybe half of them are really active in it. But the option of doing the community thing or a team thing is kind of neat. That’s something you added on a little later, right? Yes.

First it was the individual user, and then we got some feedback, and we quickly saw the opportunity for teams to use it. And we are very excited to have you on. And you guys did like an exercise a week. Right. And then code and then compare solutions or something.

No, I’ll just go around and pester people that I haven’t done it for a while, but it’s fun. And then you’re recently adding sort of the same sort of idea, but interviewing. Right. So instead of a take home problem, it’s a place for people to submit code challenges or little exercises as part of an interview process.

Yeah, correct. So first step on the enterprise tier. But then we went to Python, and all of a sudden, a lot of people were asking like, can we use your platform for interviewing? And we kind of had that. But like bit under, snowed underrepresented. So we made it a whole separate tier. And yes, the recruiter can now go in and add their own bytes. But then we got some feedback, like, well, we don’t really have time to write bytes. So now they can copy our existing exercises and use those for interviewing so they can send out interview links, and the candidates can then code in their own time before even stepping into an interview. So that takes the pressure off and lets people code how they would normally do it without people watching over their shoulders, probably, and using stack overflow and the interviewer has a submission page and then can do kind of a prescreening because maybe there are 20 interviewees who have done the exercise by those private interview links, and then they can compare all the submissions and vet people already before interviewing and see if they can solve the exercise and if the code makes sense.

It’s pretty neat. I’m looking forward to being able to use that. I haven’t needed to hire anybody since you’ve had that out, but it’s kind of a neat idea.

Julian wrote an article on the whole whiteboarding thing and we did a bit of research and it takes a lot of effort and cost to hire talent.

Thank you, PyCharm, for sponsoring this episode. If you use PyCharm already but haven’t updated recently, you’ll want to try out the new update. The 2019.2 update was released near the end of July, and it’s amazing.

Something I’m really excited about are the changes to the Jupyter notebook support. They added more run controls. You can run the current cell, of course, but you can also run and rerun all of the cells. You can also run all the cells above the selected cell or all the cells below it. It’s very useful for working on and debugging changes and exploring data. It also looks a bit nicer. I didn’t know I needed that little extra space between cells, but now that I have it, I like it more. And more work is being done in notebooks in all sorts of fields, and people like me want all of the power of notebooks with all of the power of a great editor. Plus, I like my little editor tweaks and dark background and keyboard shortcuts. I don’t want to give those up just to use a notebook. I’m so glad I can create, edit and run Jupyter notebooks right in Pie PyCharm. There’s also tons of other new features too, so check out the new update, try the new notebook support and tons of other great productivity enhancing goodies by going to testandcode.com PyCharm before August 30.

This looks complicated. I wouldn’t know how to make one of these.

How much of the guts behind what I see on the web page are you willing to share what’s this built on?

Okay, interesting. So it’s a big monolithic app in Django two something pretty new. So it’s a Django app. The front end we use Movie CSS, kind of a bootstrap framework thing. And the code editor you see is ace some Javascripting. So that’s the front end, but the whole code execution, that’s where it gets interesting.

Ideally, you don’t want to have people writing like random code on your server from a security aspect.

Right.

But also, as I mentioned earlier, the front end in the browser Python tools were not really they didn’t support the full standard library or we couldn’t get external libraries, obviously.

So we needed code execution somewhere else. So we use AWS Lambda as a sandbox to run the user code and get back the results. So technically works is when the user hits save and run code, the code is saved in temporary file. The tests are also saved in a temporary file exposed by an API on the platform we call AWS gateway, which then grabs those files, runs the Lambda, and gets back a result, and then the user on the front end sees the pass or fill score, et cetera. So, yeah, it seems like complex and there’s a couple of calls, but you’ve seen it in action. It’s actually pretty fast, right. You get your save and run and you see the response one, 2 seconds most yeah.

It seems totally reasonable.

But there are a couple of calls going on, actually. Yeah.

The code isn’t running on your server and it’s not running in the client, it’s running on an AWS something.

Yeah.

Okay.

Yeah. Like separate instance.

That’s kind of cool. How much of it is, I guess it’s hard to estimate, but you’re using a lot of pieces that were like Django and other pieces that are off the shelf sort of things.

Yes. To continue with the deployment. So the app then the Django app is deployed on Heroku. We use Postgres as a database. We use Sanskrit for emailing. We use Heroku scheduler for quantums, like periodic things we need to check, like subscriptions.

There are some interesting libraries like DJ database URL for managing databases. Of course, we just request use even Pillow to generate our certificates, because users get certificates after they finish 100 bytes, or a couple of them. We use Selenium for testing, but I guess we touch upon that later.

Oh, yeah.

Okay.

All the tests are written in Pi test. I guess we come back on that later as well. And yeah, the important thing, all the exercises and the whole stack runs on Python three seven, with the exception of Pandas, for which we had to use three six to make it work in Lambda. But three six is still pretty new. And we also use a couple of things more GitHub API because we started with a GitHub login, and later we implemented a regular login and we used Appear for integration with some other APIs. That’s pretty. And we started using retail docs for the documentation. So we’re using all kind of, as you say, off the shelf stuff, mostly free or very reasonably priced. So that’s pretty exciting.

You said this is hosted on Heroku.

Yeah.

Neat. We did talk about brushbyte testing a little bit. So this is a side thing. This is not your day job, right?

Yeah, a side thing.

So you can’t spend a ton of time on it. Or maybe you can.

I’m assuming testing is a serious part of this to make sure that you don’t have to spend a lot of time debugging.

Yes.

Let’s talk about that. If you want to make a change, you’re changing something. How do you verify that things are still working?

The main test suite is Selenium, so we have a bunch of test cases which if we make major changes, we run. Actually, I can link YouTube video where you see all those tests running in parallel. You see like 20 terminals, 20 browser windows going through all the stuff. It’s pretty cool, actually, but yeah, ideally you want to move a little bit more into unit testing because Selenium test can be a bit brittle. Like there are some dependencies. We have a bunch of users, for example, and they use GitHub login. That was our primary authentication system from the start. And sometimes get up likes to do like an extra verification, I don’t know, some security thing, and then your test breaks because it’s all automated. Right. So you cannot just sit there and go to the headless browser and just quickly click on that button. So it’s great. Overall, it’s doing a lot of testing, but I want to add some more unit testing to make it faster and just test more, smaller things. Now, of course, we also do manual testing. If we roll out a new feature, we do some quick testing also to get a bit of feel of how the user experience is. But yeah, the main automated thing is sitting in for now, a new feature.

You’re going to both try to put something into a Selenium as well or some other test, but you’re manually testing it also. Yeah, but then all of the existing features, you’re relying on your test suite to make sure all the existing functionality works.

Yeah. There’s a lot of regression testing, which is nice and which has saved me a few times.

Yeah.

I guess it’s unexpected that you’re using so much Selenium because, like you said, making sure all the workflows work and everything is dependency. If you could stub out the login, something, for instance, that could save you quite a bit, right?

That’s correct. We made pictures for the whole login thing, and there are some flow that user comes in and accepts GDPR that’s like all in a fixture. So you can just reuse that in every test. So that definitely saves work. Okay.

Are you testing the database and the back end separate from the front end, or if it’s through Selenium, you’re just testing all of it together?

Yes. And it was a bit tricky to get the Selenium and database working. I have to look at the code. I actually do that Selenium.

Most testing is just like going through the flow and the automated user, so to say, is doing things. And then you just look at the resulting web pages showing what you would expect, which then kind of means that the database is doing its thing. Right. Somebody subscribes you go to the inbox with the welcome message. But the fact that the welcome message is there means that the whole operation works right now.

Are you doing this on just a local computer or are you doing it on a test server? Local machine?

Yeah, just on my Mac. Okay.

And then testing is part of the user experience, too. The challenges have tests associated with them?

Yes. All of them.

That’s all my tests. Yeah.

And I’m thrilled with that. And I’ve given some feedback to you guys for some of the testing.

I wanted to thank you for the feedback you have given in the past because we did improve a couple of tests based on what you told us.

One of the fun things is sometimes I look at them and I think all three of those could probably just all be one test with parameterization, but it might confuse the user a little bit.

He ended up using parameterize or how it’s called quite a lot because it’s so nice. Right. It’s taking a lot of repetition away. And yeah, it’s true. How would the user perceive it?

Yes. Have you gotten any feedback from anybody about that?

I remember Julian saying in the show that we get feedback all the time.

I understand this. This test can be written better, so it’s kind of a Kaisen thing that we make a lot of improvements as we go. So we’re very happy that all the exercises are in a database in one central place where we can edit them so they’re not emailed out, so they’re sitting in an inbox and you cannot do anything about it. So server side. So whatever we update is live on the site in an instant.

Every day we make tweaks based on feedback, making it better. And a lot of it’s description because it’s hard to kind of always describe a problem and that people understand it. But some of the changes are also in the test.

Seems like all of the pieces getting the payment processing and having users and building up the challenges and all the moving parts seems daunting. What was the hardest thing for you that you didn’t expect it to be as hard as it was?

That’s a good question.

It does seem complex. It is complex. I think the hardest thing was to get started, like, not over perfecting when you start, because when we started, we actually started to make some sort of tool around the block challenges. So it was all GitHub, login and all focused on how people could pull requests at work and all tailored towards the block challenges. And then we kind of organically grew into the whole exercise, platinum, the code in the browser thing. The hardest part, I mean, funny enough, extending it these days with new features is not that hard. I think if you have your models in place and like sensible defaults with Django, it becomes easier relatively. So I think the initial structure, the initial framework is the hardest.

Are you using the built in Django RM?

Yes. That’s awesome.

Is there any upcoming features coming in, the code challenges that you can share or that you’re excited about?

I have to mention recently launched rolled out profile pages. So with the recruiter stuff, we also want the other party to benefit so that people applying for jobs so they can have their own profile page where you can see how many points they got, how many bytes they sold. We even have coding calendar, so they kind of bit of like GitHub that you see all the Greens on the calendar. So how active somebody is.

Nice profile pages and last week learning paths. So we got to 200 exercises. And yeah, a lot of people start to ask, like, how can I learn data science?

What is relevant for OLP object oriented programming?

We just start grouping those bites together into learning paths with nice ditches and stuff. And people are pretty excited about that. And you mentioned that’s also fairly new. But yeah, you want to know about the future. Of course, the one thing you’re not working on is the new VI bites, which will be a new collection of 20 bytes, because some people come into the platform and they have a bit of trouble getting even to the beginner bytes or the intro bytes because of that possibly steep learning curve of a challenge in itself. I mean, going through a description of 20 lines plus writing code in a function, plus making sense of the bytes output. So that’s kind of a lot of to take in for some newer programmers or people that are just starting out with Python and you always have to send them away to another resource. And we don’t like that. So we’re bridging that gap now and we’re making some very simple bites so that people absolutely need to type and can go through those 20 exercises, which will be way more detailed than the descriptions are really hand holding and make them ready for the serious bites. So that’s one concrete thing we’re working on. Now.

One of the things I’m kind of jealous about that I think is brilliant, is you don’t actually teach people how to do this stuff self service.

Right.

People can learn by doing the challenges, but you don’t have associate resources to say, this is how you learn how to do this. People have to use like Google or whatever to figure out how to get that done.

Yes. And some people hate that. Right. And some people love it because they are forced to look at these resources. And yesterday somebody said, that’s really what I actually need to learn as a developer. I need to kind of get up to speed with any technology in getting the ground running, and that’s what the platform is teaching me. Right. So, yeah, that’s two sides.

I was just thinking that that’s definitely a job skill on the job. You’re not going to have how to solve this problem. You got to figure it out with whatever you’ve got.

Often myself. Right.

And just expected of you.

I did not know about these learning paths, and I’m excited about these.

That’s Brandon.

It’s so cool. We’ve got like data analysis, path, object oriented programming, web scraping. Web scraping, that looks fun.

Yeah, I was surprised you could even do that. But as we now support external libraries, turned out that we had a couple of buy some beautiful soup and request and yeah, we had the materials. I mean, it’d be 200 exercises. It becomes easier.

Well, it’s a great idea because like you said, you’ve got like 200 exercises. Which one should I do? But when you break it into a path, you’ve only got like a dozen or so in a certain path. It’s not too daunting.

Yes. And another thing I want to put in the bucket of the future or already future in the present is that we’re working with a couple of Bites authors. So we’re not only running the Bites ourselves anymore because we’re spread too thin, as you know, and we don’t know everything. Of course, we only know our bit of Python, so we’re getting some more specialties in one in particular, network programming we don’t know anything about, but we do want to offer on the platform. We’re working with a couple of folks and they’re adding buys as well.

That’s a great idea.

Yeah.

Cool. Well, thanks so much for coming on and telling us about all of this cool resource. I’m excited to see where it goes. I think it’s also an exciting example of what people can do, just as a side thing to help out the community. This is also an example of look at this and you go, oh, my God, I could never make this. But they didn’t make it like this to begin with. It started smaller and it’s built up over time. So that’s cool.

Thanks a lot for having me. I really appreciate it.

All right, thanks. We’ll talk to you later.

All right. Cheers.

Thanks to Bob for that fun discussion and for bringing us a cool tool to practice Python. Thank you to PyCharm for supporting the show. Don’t forget to go to testandcode.com/pycharm before August 30 to get your four month free trial of PyCharm Pro and get your work done faster with a great editor. That link is also in the show notes at testuncode.com 83. Thank you also to Patreon supporters. You’ve been with me through thick and thin, and it means the world to me. That’s all for now. Go out and maybe start something small, release it and grow your skills with your project like Bob and Julian have. But whatever you do, make sure it’s well tested with Pi test of course.