Week 6: Final Project

[MUSIC] Congratulations on making it here. You’re almost at the end of the course. It’s been an interesting and
rewarding journey don’t you think? Along the way you’ve learned the basics
of Python syntax, including functions, conditionals, and for and while loops. And you’ve learned how to use the most
common data types, like integers strings, lists, and dictionaries. You even learned about
object-oriented programming. Now we’re going to put all this
knowledge together to solve more fun and exciting problems. We’re going to approach these new
challenges like they’re real-world problems. We need to solve with a script. By doing this we’ll see yet
another example of how these programming skills can make the work we do in our
IT jobs faster and more efficient. In the next videos, we’ll check out how to
go about solving a more complex problem by writing a script from the ground up. To do that we’ll go step by step
using a recommended way for dealing with more advanced challenges. Back at the beginning of the course, I told you a little bit about
the first Python script I ever wrote. It all started with a problem, my teams on
call person was getting paged too much. Being on call is drawing
the ultimate short straw. Whenever an issue springs up
the person on call needs to be there to put out the fire. It’s an exhausting challenge. So to help alleviate some of the stress
we wanted to build a better monitoring dashboard. Getting it done took a lot of refactoring,
debugging, and testing. It wasn’t easy, but thankfully I
didn’t have to start from scratch. I had the help of my teammates and many thousands of people who posted
similar struggles on the Internet. When the dashboard was finally up and
running, the on-call person wasn’t the only
one breathing a sigh of relief. To start solving our problem, we’ll first
look at the problem statement where we’ll get an understanding of what we need
to do and the inputs and outputs for the script we’ll need to write. Then we’ll do some research. We’ll think about how we can tackle the
problem with the tools already baked into Python. Remember that we always want to
avoid reinventing the wheel. No matter how tricky and
intricate the challenge appears, chances are that others have
solved a similar one before. So it’s valuable to spend some time
tapping into the resources that exist to help us solve our problem. Once we know what we need to write and
what we can use it to do, we’ll do some planning. We’ll think about what data types
will be useful for our solution and how we’re going to operate on them. Finally, we’ll do the actual
writing of the script and then we’ll check that the code
does what it’s supposed to do. Sound good?
When we take the structured approach to tackling problems there really isn’t
a challenge too complex to solve. So let’s get started.

Imagine that you’re
an IT specialist working in a
medium-sized company, your manager wants to create a daily report that tracks
the use of machines. Specifically, she wants to know which users are currently
connected to which machines, it’s your job to
create the report. In your company, there’s
a system that collects every event that happens on
the machines on the network. Among the many events
collected it records each time a user logs in
or out of a computer. With this information, we want to write a script that generates a report of which
users are logged in to which machines
at that time. Before we jump into
solving that problem, we need to know what
information we’ll use as input and what information
we’ll have as output. We can work this out
by looking at the rest of the system where
our script will live. In our report scenario, the input is a list of events, each event is an instance
of the event class. An event class contains the
date when the event happened, the name of the machine
where it happened, the user involved,
and the event type. In this scenario, we care about the login and logout event type. All right, that’s good to know. But we need to know exact
names of the attributes, otherwise, we won’t be
able to access them. The attributes are called date, user, machine, and type. The event types are strings
and the ones we care about are login and logout. With that we should have enough information about
the input of our script. Our script will receive a list of event objects and we’ll
access the events attributes. We’ll then use that
information to know if a user is currently logged
into a machine or not. Let’s talk about the output. We want to generate
a report that lists all the machine names
and for each machine, lists of the users that
are currently logged in. We then want this information
printed on the screen. We’ve been tasked with
generating a report and we can decide exactly how we want that report to look. One option would be
to print the name of the machine at the beginning
of the line and then list the current users on separate lines and
indent it to the right, or we could print the machine name followed
by a colon and then the usernames separated by
commas all in the same line, and we can probably come up with something even more fancy. When formatting a report, it’s easy to get caught up in the making it look good part. I’ve fallen into that
trap but what really matters is how well the
script solves the problem. So it’s better to first focus
on making the program work. You can always spend time making the report
look nice later. Let’s keep it simple for now and we’ll go with the
approach of printing the machine name followed by all the current users
separated by commas. Okay, we now have a pretty good idea of
what we need to do. We’ve identified our problem
statement which is we need to process a list of event
objects using their date, type, machine, and user
attributes to generate a report that lists all users currently
logged into the machines. We’re off to a great start. The next step we’re
going to do is some research to work out how
to best actually do this.

Okay. So we have our
problem statement which helps us understand the problem
and focus our approach. We know we have to input a list of event objects and evaluate these objects
attributes to output a report of all the users currently logged into a machine. Now it’s time for
step 2, the research. We’re going to consider
all the tools we have available to help
us solve the problem. To find out which users are currently logged into machines, we need to check when they logged in and when
they logged out. If a user logged into a
machine and then logged out, they’re no longer logged into it. But if they didn’t
logout out yet, they’re still logged in. I know we’re stating
the obvious here, but in programming, it is super important to be clear
on the parameters. Also, knowing this tells us
that to solve this correctly, it’s vital that we process the events in
chronological order. If they’re not, we can get
the logout event before the corresponding login event and our code may do
unpredictable things, and no one likes
unpredictable code. So how do we sort
lists in Python? We’ll need to do some research. Type sort lists in Python into your favorite search engine
and you’ll get a bunch of results that mentioned
the list sort method and the sorted function. The difference between
these two options is that the sort method modifies
the list it’s executed on, while the sorted function returns a new list
that’s been sorted. Apart from that, they
work the exact same way. Let’s check out this
difference in action. First, will create
a list of numbers and call the sort method
to sort the list. You can see here that the elements of the
list have been sorted. Let’s try a different example now using the sorted function. We’ll create a list of names. Then we’ll print the output
of the sorted function. Let’s print the original list again to check that
it didn’t change. So you can see that the
original list wasn’t modified. The sorted function
returned a new sorted list, but the original
was left untouched. Nice, we now know how to
sort things in Python. For this problem, it’s fine
to modify the original list. So we’ll use the sort method. But wait, see how both these options sorted the
list alphabetically? That’s the default
approach Python takes. But what if we wanted to organize our lists by different criteria? Again, if we take a look at the documentation
we found online, we’ll see that the sort method can take a couple of parameters. One of these parameters
is called key, and it lets us use a
function as the sorting key. Let’s try this out on
our list of names. Instead of sorting
alphabetically, we could sort by the
length of each string. Do you remember which function
we can use to do that? Yes, we can pass the len
function as the key. All right. We now
know how to order elements of a list based on the return value of a function. In our report scenario, we know that our elements
will be instances of the event class and we
want to order by date, which is an attribute
of the event class. One way we could do this is
to write a function called get_event_date which
returns the date stored in the event object. We could also create
this as a method in the event class if we had
access to modifying the class. But since we’re working
with a bigger system that generates these events, we will assume that we can’t just add a method to the class. So we’ll create our own function instead. How does that sound? Isn’t all making sense? Remember that there
are various paths we could take to
solve this problem. But some are better than others. So it’s important to understand why we chose the options we did. Feel free to take some
time on your own to explore the possibilities and understand what we’re doing. In the next video, we will dive into our
plan to build our script.

Okay. You’re doing
great with this so far. We’ve already defined
our problem statement and then we researched options to figure out what tools we have available and which
are best for the job. Now it’s time to
plan our approach. So we know that our
input will be a list of events and we’ll
sort them by time. Each event in that list will
include a machine name, a username, and tell us whether the event is
a login or a logout. We want our script
to keep track of users as they log in
and out of machines. So how can we do this? Let’s think about
what we’ll do for each event and see if we can figure out the best strategy. When we process an event, we’ll see that someone
interacted with a machine. If it’s a login, we want to add it to the group of users logged into that machine. If it’s a logout, we
want to remove it from the group of users
logged into the machine. In this scenario,
it makes sense to use a set to store
the current users. Adding new users at login time and removing
them at logout time. Great. But if the current users of a given machine
are stored in a set, how do we know which set corresponds to the machine
we’re looking for? The easiest way to
know this is to store this information
in a dictionary. We’ll use the name of
the machine as the key and the current users of
that machine as the value. So for each event we process, we’ll first check in the dictionary to see if the
machine is already there. We need to check this
because it could be the first time we’re processing an event for that machine. If it’s not there, we’ll create a new entry. If it is, we’ll update the existing entry with the action corresponding
to the event. Which means we either
add the user if the event is a login or remove
the user if it’s a logout. Once we’re done
processing the events, we’ll want to print a report of the information we generated. This is a completely
separate task. So it should be a
separate function. This function will receive the dictionary regenerated
and print the report. It’s important to have
separate functions; to process the data and to
print the data to the screen. This is because if we want to modify how the report is printed, we know we only need to change the function in
charge of printing. Or, if we find a bug in
our processing the data, we only need to change
the processing function. It would also allow us to use the same data processing function to generate a different
kind of report, like generating a PDF
file, for example. Yay, we know what we need to do, how we need to do it, and how
we’ll structure our code. Now we can get into
the meaty stuff, actually writing the code.

We’ve come a long
way to get here, so let’s quickly rattle
off what we know so far. We know that we need to process the events to generate a report. We know how to sort the list
of events chronologically. We know that we’ll store the data in a dictionary of sets, which we’ll use to keep track
of who’s logged in where, and that we’ll have a
function that generates the dictionary and a separate one that prints the dictionary. I think that’s everything.
Know what that means? We’re finally ready to
write our code. Here we go. Let’s start by defining the helper function that
we’ll use to sort the list. We’ll use this simple function as the parameter to the sort
function to sort the list. Now, we’re ready to start
coding are processing function, which we will call current users. The first step is to
define the function. Inside the function,
we’ll first sort our events by using
the sort method, and passing the function we
just created as the key. Now, before we start iterating through
our list of events, we need to create the
dictionary where we will store the names end
users of a machine. Now, we’re ready to iterate
through our list of events. Next, we want to check if the machine affected by this
event is in the dictionary. If it’s not, we’ll add it with
an empty set as the value. Now, for the login events, we want to add the
user to the list, and for the logout events, we want to remove
users from the list. To do this, we’re going to use the add and remove methods, which add and remove
elements from a set. Once we are done iterating
through the list of events, the dictionary will contain all machines we’ve seen as keys. With a set containing the current users of the
machines as the values, this function returns
the dictionary. We’ll handle printing in
a different function. Nice. We now have the
dictionary ready, and we need to print it. For that, we’ll create a new function called
generate-report. In our report, we want to iterate over the keys and values
in the dictionary. To do that, we’ll use the
method items that returns both the key and the value for each pair
in the dictionary. Now, before we print anything, we want to ensure
that we don’t print any machines where nobody
is currently logged in. This could happen if a user logged in and then logged out. To avoid that, we tell the
computer only to print when the set of users has
more than zero elements. Now, we said earlier that we want to print the machine name, followed by the users logged into the machine, separated by commas. Let’s generate the string
of logged in users for that machine using
the method join. Now, we can generate the string we want using the format method Yeah, we’ve written
all the functions we need to tackle our problem. Did everything makes sense? This is a great moment
to pause and review the videos for each
step in our approach, from problem statement
to writing the code. Make sure it’s clear, not just which function we’re using, but why we’re using it. If anything is a little fuzzy, remember the
discussion forums are always there for you
to ask for help. It’s about to get exciting
in the next video. We’re going to execute this
code and see if it works. I’m feeling good about it. Let’s put our code to the test.

[MUSIC] We’re almost done solving our problem. We’ve written the code that solves
our problem statement after following our research and plan. We’re now going to put all of our code
in a jupyter notebook, execute it and see what happens. This is what our code currently looks
like as we wrote in the previous video. To check that our code is doing
everything it’s supposed to do, we need an event class. For this scenario,
we’ll use the very simple event class. Okay, we have an event class
that has a Constructor and sets the necessary attributes. Using this Constructor, we’ll create
some events and add them to a list. Okay, we’ve got a bunch of events. They’re currently unsorted, they affect
a few machines and include some users. We’ll feed these events into our
function and see what happens. Everything is now ready to go. Drum roll, please. [SOUND]
Great, our code correctly created a dictionary
with the machine names as Keys. There’s one empty set and
two sets with one value. Let’s now try generating the report. [SOUND] Success, our report correctly skipped to
the one machine that had an empty set. That’s great. In the world of IT, there’s a bunch
of other things that could happen. What happens if we come
across an event for a user logging out that
had never logged in? What do you think we should do then? We’re going to try and
figure this out in the next exercise.

As we did with the earlier exercises,
we’re going to use a jupyter notebook for the graded assessment. But this time once you’re
done with the exercise, you need to submit your code for grading. In this video, we’ll guide you through what you need
to do to submit your assessment. It’s pretty simple. You’ll start by running the notebook and completing the exercise just as
you’ve done with the other ones. There’s a bunch of stuff to fill in and detailed instructions that
guide you through the process. Take your time and make sure you filled in
all the functionality that’s asked for. Once you’re done, you can submit the assignment by clicking
the submit assignment button at the top. When you submit your assignment,
you’ll see a pop-up that gives you a link where you can find
your assignment results. If you close the pop-up without clicking
the link, you can find the results by going back to the course interface and
navigating to the next element. The grader can take a few
moments to execute your code and check if you’ve done everything correctly. If you come to this page and it’s not done
grading, don’t panic, grab a snack and by the time you come back it’ll be done. If your submission was incomplete or had some mistakes, the grader will
let you know that you didn’t pass. But don’t worry, you can always try again. Go back to the notebook and make sure that
you’re following all the instructions and complete all the missing functionality. Once you’re ready, you can try again by clicking
the submit assignment button again. And that’s all there is to it.

I’m an SRE and in Google an
SRE, there historically, there used to be very few women, but leadership really
cared about diversity. So they made a lot of efforts of hiring more
diverse candidates, not just women, but people from different backgrounds,
different minorities, and by now is really
has a lot of diversity, and it’s noticeable
that when you have more diverse point of
view at the table, you get better results. I was almost always the
only woman in my class, and while I was at university
I had to endure a lot of discrimination and more keying and people thinking that
I wasn’t good enough, that I didn’t belong. When we had to do assignments, they would have me do the documentation instead of
doing the actual assignment, but I know that I belong
here and that I am amazing, it doesn’t matter what
the others think. So when creating the
content for this program, I really tried to keep
diversity in mind, and I tried to
incorporate it in many of the examples that are in
the blueprint courses. As I was writing the
scripts of the courses, I never assumed that the learner has the same micron as me. I tried to put myself
in the shoes of different people and make
it all make sense for them

Wow, first off, a huge congrats. You’re about to start the final
project of the course. Amazing job making it all the way here. I hope you’ve been having as much
fun as I have on this journey. We’re going to be working on our final
project using Jupyter notebooks. You might be starting to feel
pretty confident using them. But remember if you have any
issues you can always ask for help in the discussion forums. Okay, before we dive in, we’re going to
chat a little bit about what you’ll be doing for the project,
it’s going to be really fun. The goal of the project is
to create a word cloud. A word cloud is an image that’s
made up of different sized words. Usually the sizes of the words
are determined by how many times each word appears in a specific text. To create the image itself, we’re going to use an external Python
module called creatively Word cloud. [LAUGH] Your job is to create a script
that would go through the text and count how many times each word appears. We’ve done this a few times already,
any ideas how we should tackle this one? If you’re thinking of using a dictionary
to count how many times each word appears, then. Ding ding ding, good answer! You’re going to prepare a dictionary and
use that as a parameter for the word cloud module,
not too tricky, right? I think you can handle a little more,
so two things you have to watch out for. One, punctuation marks, before
counting the frequency of the words, you need to make sure that there
are no punctuation marks in the text. If you don’t, a string example
with a comma at the end would be different from a string
example with a dot at the end. So before you put words
into the dictionary, you have to clean up the text
to remove any punctuation marks. And the second thing we want to
keep our word cloud interesting. Certain words in our
language crop up a lot and if we include all of these we’re going
to get a pretty dull word cloud. Think about words like, a, the, two or if. They usually appear
a whole lot in text but aren’t too relevant to
the text’s overall message. We want our Cloud to show words that
are relevant to the text we’re using for the input. So you need to find a way
to exclude irrelevant or uninteresting words when
processing the text. For the input,
you’re going to upload a text file. You can choose any text file you like for
your input. It could be the contents of a website,
a full novel or even everything that one
author has ever written. You just need to make sure
that it’s one text file, so that it can be processed by the code. Okay, before jumping into the project, remember you can re-watch this
video If something isn’t clear. Yep, I’m starting to sound
like a broken record, but this time it’s extra extra important. This final project is the real test of how
much you’ve gotten your head around and can highlight areas you
need to brush up on. So we want you to be super clear on
what you need to do on that point. You’ll find an overview of what you
have to do in the next reading. Can you guess the best way
of tackling this problem? Yep, you got it, our step-by-step
approach that we outlined earlier. Understand the problem statement,
research available options, plan your approach,
write your code and finally execute. Okay, feeling good? Ready to go? Remember, you know this stuff and
you’ve totally got it.

Congratulations! You’ve made it through
the entire course. These weren’t easy
concepts to learn. I want you to think
all the way back to when you were just
starting these journey. You remember what you
were feeling when you watched to those first videos, maybe a little nervous,
terrified even, excited, probably all
these emotions at once. You tuned in with me, watched all these videos, and kept going when
it got complex. You should be proud of yourself. Take a moment to reflect
on where you are now. You’ve gone from having little or no knowledge of programming, to being able to write all
kinds of complex functions. You’re using conditionals, loops, strings, lists, and
even dictionaries. You even created
your own objects. You put it all together to
write your very own program, applying a process
which you might use in your day-to-day IT role, and hopefully you had as much fun doing it
as I did teaching you. It’s impressive that you’ve
mastered all the stuff, and I hope it’s
just the beginning of your Python journey. Building a successful
career in IT calls for perseverance,
curiosity, and grit. Three qualities you’ve proven to have heaps of by
making it this far. It also requires
skills and knowledge, and basic Python is definitely a powerful tool in
your IT toolbox. Knowing how to write scripts, will set you apart from others as you look to advance
in your career. I bet you’re now seeing tasks
all around you which are sparking ideas on how you could automate them
with a script. The possibilities are endless, and it’s just the start. Remember what we said in
one of our first videos, a journey of a thousand miles
begins with a single step. There’s still a lot of
exciting things you can learn. We hope you will join us in the next course where
we will be learning all about how Python interacts with the computer’s
operating system. To get a taste of
what’s in store, stay put for the next
video where we will chat with Roger, your
next instructor. But for now, I want to
wish you best of luck. I look forward to seeing you
and your code out there.

You’ve made it through
the first course, but your journey with
Python is just heating up. In our next course, my friend and colleague, Roger, is going to be your instructor, as you learn how Python interacts with the
operating system. You’ll build on all the
skills you learned here and your programming is going to get a little more sophisticated. Hey Roger, so what can learners expect from
the next course? Hey Christine, and hello to
all the learners out there. I’m super excited
about the next course. We’re going to
cover how to set up your own developer
environment in Python, and in no time, you’ll start feeling
comfortable using codes interact with the system. We’ll also manipulate files and processes running on the OS, and dive into RegEx which is a super powerful tool for
processing text files. You’re even going to write
a script that might be similar to a task you’d
be assigned at your job. But personally, my favorite part the whole course is definitely where we talk about the Linux OS, and it’s not because it’s the
primary OS I use in my job. Linux opens up a whole world of customization
and configuration, and I find it really interesting. We’ve got a lot of powerful
and fun concepts coming up. So don’t miss out. I’ll see
you over in the next course. Thanks Roger, and thank you, everyone of you, for tuning in, and hanging with
me in this course. I’m not crying, you’re
crying. Bye for now.