Module 1: Getting Your Python On

Hi there, and welcome back if you joined
us from the Python Introduction Course. And if you skip that course, and this
is the first time we’re meeting, well, welcome aboard. If you did join us for
the introductory course, you’ve heard this before, and no doubt you’re going
to hear it again. When you work in IT, computer
programming skills open up an incredible amount of opportunity. You’ve also
heard us use the metaphor of the toolkit that every successful career in IT is
built upon a toolkit of skills, experience, knowledge and curiosity. Python scripting is a powerful tool in
your IT toolkit. And you’re here because you’ve taken
the first steps on your Python journey. By now, you’re starting to understand
programming concepts, and you’ve seen how an IT specialist might create
automation to save time and cut down on the stakes. My name is Roger Martinez,
and I’m a system administrator for Hardware at Google. The team I work on
maintains and monitors servers that Google hardware engineers use to design
and test silicone chips that Google makes for their Pixel phones. I’m really excited to be your
instructor for this course. As you know, this program has been
developed and designed exclusively by Google. Each course takes place at a
different campus location to bring you some extra Google goodness. And in case you’re wondering, no, we
aren’t actually orbiting a distant galaxy. Although, we are trying to
bring you a universe of knowledge about Python. Don’t worry, the curriculum is
stronger than a lot of my jokes. Throughout this program, you’re going
to meet other Googlers besides me and the other instructors. Many of them
started their careers in IT support just like me, learned programming and
advanced onto more technical roles. They’re excited to share their stories
with you about how they’ve built their IT careers through persistence and a
commitment to learning new skills. Some of them will also share more
detail on their jobs and what their day-to-day tasks look like. Before we dive into the course, I’d
like to share with you a story to give you some insight into why I’m excited
to be here, teaching you this course. Years ago when I started college in New
Jersey, I really didn’t know what I wanted to study. A friend of mine
recommended that I check out the IT program. I’ve always liked working on
computers but I had no idea that a career in IT was even a possibility. From the moment I walked into that
class, I felt so out of place. No one really looked like me. No one sounded like me, and no one
seems to come from a background similar to mine. I dropped the class the next
day, and I never went back. In hindsight, I kind of regret that
decision. If I would have stayed in that class,
then maybe the next Latino person or other minority that walked in would
have felt like they belonged. Diverse ideas, experiences and
backgrounds are crucial to IT, to any industry really. And diverse
representation is so important to showing the next generation that they
belong in a field and that they can thrive there. So that’s why teaching
this course is so meaningful to me. Maybe it will, in a small way, help
more people walk through that door and never turn back. Okay, enough about me. Let’s talk about the course. So, what’s
ahead? By the time you finish the last video
of this course, you’ll be able to manipulate files and processes on your
computer’s operating system. You also have learned about regular
expressions of very powerful tool for processing text files, and you’ll know
how to use the Linux command line. And this might feel like a stretch
right now, but you’ll also write a program that processes a bunch of
errors in an actual log file, and then generates a summary report. That’s a super useful skill, and by the
end of this course you’re going to know how to do it. We’ll kick it off now by
exploring how to execute Python locally and organize, and use code across
different Python files. We’ll then learn how to read and write
different types of files, and use sub-processes, and input streams. We’ll also dive with the bash scripting
and regular expressions. Both very powerful tools for anyone
working with systems. We’ll even touch on automatic testing,
which allows us to automate how we check if our code is correct. To finish, we’ll put all of this
together using the tools that we’ve acquired to process data and generate
automatic reports. We’ll also explain how to set up your
own developer environment in your machine. And we really recommend that
you do this. You need to have administrator rights
to install software, or you’ll have to ask the administrator to install it for
you. Setting up your developer environment
is a key step in being able to write and deploy powerful automation tools. So we strongly advise that you do this
and get comfortable with working with it. In this course, we’ll use quick
labs, which is an environment that allows you to test your code on a Linux
virtual machine. This lets you experience real world
Linux scenarios where you will need to write code to solve problems. We’ll use Linux for our examples and
exercises in this course. Linux is an industry standard for
running servers, which is why we chose it. At some points though, we’ll talk
about how things are done on Windows or macOS to give you a broader picture. While we’ll be using Linux a lot, you
don’t need to run Linux in your machines to follow along. What you do need to do
is make sure you’ve got Python installed. You also need to know how to
install additional Python modules. But don’t worry, we’ll walk through all
that later on. You’ve already learned the building
blocks of programming. In this course, we’ll examine different
tasks that you might come across in your day-to-day activities. And we’ll learn
how we can solve them through programming. One difference I bet
you’ll notice real quick, if you joined us for the introduction to Python
course, is that we’ll use a lot more modules than before. And when I say
modules here, I’m not talking about the parts of this course. In this context,
I mean the Python modules that we can use to get extra functionality for our
scripts. Please remember, there might be some
complex topics and videos that may not 100% sink in the first time around. That is totally natural, take your
time. Rewatch the videos a couple of times if
you need to. I promise you’ll get the hang of it. Also, remember that you can use the
discussion forums to connect with your fellow learners and ask any questions
that you might have. We’re about to go on a fun and
fascinating journey together into the universe of Python. So, let’s go.

Welcome to the next chapter in your
programming journey. In the first course of the program, we
built a foundation in Python. You’ve done an amazing job getting to
this point. Now, it’s time to take the next step. We’re going to build on that
foundation, to boost your programming toolkit with even more useful tools. Are you excited? Are you a little
nervous? Don’t worry about a thing. As always,
we’re here to guide you and help you build your confidence step-by-step. Are you ready? Let’s do this. To really get the most
out of this course, we strongly recommend that you have Python
installed on your machine. Having Python at your fingertips like
this, lets you understand how to run Python programs, how to install
additional Python modules, and how it all comes together. So we’ll devote
some of the next videos to guiding you through setting up your environment. We’ll then use your newly configured
Python environment to run Python scripts locally. Discover how to organize code
into different files, and talk about what coeditors are for, and how to
choose what tools to use. After you get set up and familiar with
your environment, we’ll dig deeper into some automation concepts. In the introductory course on Python,
we talked a lot about automation. We saw some automation examples, and
spoke about many possibilities that come with learning the program. In this
course, we’ll keep expanding on that base. We’ll learn how to decide whether
to automate something or not, and what it takes for automation into place. To put this into practice throughout
the course, you’ll be running Python scripts on a remote Linux machine using
Qwiklabs. This tool lets you experience
real-world scenarios that simulate programming problems you’ll likely have
to tackle in an IT job. If at any point along the way you feel
lost or confused, don’t panic. You can watch the videos as many times
as you need to let the concepts sink in. Plus, you can ask questions in the
discussion forums which is one of the best ways to find extra information,
and to connect with other learners. So we’ve a lot to cover. Let’s get started.

Which of the following is NOT an open-source OS?

Microsoft Windows

Right on! The Windows operating system is a proprietary OS developed exclusively by Microsoft.

In this course, we’ll interact with the
operating system a lot. So before we deep dive into more
complex concepts, let’s do a quick rundown of what we mean when we talk
about the operating system or OS for short. The operating system is a
software that manages everything that goes on in the computer. It reads, writes, and deletes files
from the hard drive. It handles how the processes start, how
they interact with each other, and how they eventually finish. It manages how
memory gets allocated to different processes, how network packets are sent
and received, and how each programming can access the different hardware
components. Basically, the OS is a total boss and
doesn’t mess around. There are actually two main parts in
the operating system: the kernel and the user space. The kernel is the main core
of an operating system. It talks directly to our hardware and
manages our systems resources. As users, we don’t interact with the
kernel directly. Instead, we interact with the other
part of the operating system called the user space. The User space is basically
everything outside of the kernel. These are things that we interact with
directly, like the system programs and user interface. So when we say
operating system, we’re referring to both the kernel and the user space. The operating system does a lot of
things. It does so much that we need a bunch of
courses that cover all of it. But for this course, we’ll focus on
managing files and processes on the OS running on your computer. The scripts
we’ll write will all run in the user space. In some cases though, our
scripts may interact with the kernel of the operating system to get extra
information or to ask it to perform certain operations. As you probably
already know, there’s a lot of different operating systems out there. The major operating systems using IT
today are Windows, Mac OS, and Linux. The Windows operating system is
developed by Microsoft, and is widely used in the business and consumer
space. Most PCs come with Windows as the
default operating system. Mac OS is developed by Apple and is
mainly used in the consumer space. If you purchase any Apple computer,
they’ll come with Mac OS preloaded. Linux is an open source operating
system. Open source software is free to share,
modify, and distribute. Score. Linux is used heavily in
business infrastructure. Most servers in the world today are
running Linux. It’s also available in the consumer
space, although less common. Linux itself is actually the name of
the kernel originally developed by Linus Torvalds, also a total boss. Because of the evolution of the rest of
the operating system, we typically use Linux to refer to both the kernel and
the whole operating system. Today, Linux has grown into a huge
community effort with developers all over the world contributing to its
success. Because Linux is open source, a lot of
different organizations package their own versions of it, unlike Windows or
Mac OS which are solely developed by their respective companies. We refer to these different flavors of
Linux as distributions. Some common Linux distributions are;
Ubuntu, Debian, and Red Hat. If you’ve heard of Chrome OS, you may
know that it’s another operating system based on a Linux kernel. But unlike other distributions, Chrome
OS is usually considered an OS in its own right. Lastly, the Android
operating system which is used a lot on smartphones also runs a Linux kernel. You may have also heard of Unix. Unix is an operating system developed
back in the 70’s by Bell Labs. After its original release, the OS went
through a bunch of different versions with different companies releasing
variants of it. The fundamental ideas of how Linux
works today are based on the Unix principles. A lot of the tools that we
use to interact with the operating system are open source versions of
those originally developed for Unix. This is why these tools in operating
principles are usually referred to as Unix. Even though the OS we’re using is
called Linux. So Linus created Linux which is based
on Unix. It’s basically a programming
tongue-twister. Try saying it 10 times fast. The Mac OS kernel and some of its user
space are also based on a kernel and User space tools from the Unix family
known as BSD. So although the graphical interface is
extremely different between the two, the command line is actually pretty
similar. Like we said earlier in this course,
we’ll use Python to interact with the operating system. Python is a
cross-platform language. You can use it on Windows, Mac OS,
Linux, and even lesser known Unix variants like FreeBSD. It’s even available on mobile phones. Since it’s cross-platform, we can use
the same Python code to get to our goal on any operating system, whether the
goal is opening files, processing text, or managing running processes. This makes Python a great tool for IT
specialists who needs to interact with different operating systems. You can apply the skills that you
learned from one platform to all the others. So how cool is that? As we call it out before, for the
exercises in this course, we’ll be practicing with Linux computers. We’ve chosen Linux because it’s the
industry standard when it comes to servers that are managed remotely and
through automation. When it’s relevant, we’ll also talk
about how the same concepts will apply to Windows or Mac OS. In my job as a sysadmin, I mainly work
with the Linux operating system. I grew up using Windows computers and
interacted with them through the graphical interface. The very first
time I used Linux was when I was a teenager and my PC crashed. It wouldn’t boot up into the user
environment. I had a lot of different files and docs
I didn’t want to give up. So I did some research and on a
friend’s computer, I burned a portable Linux OS called slacks onto a CD. I got my PC boot up from there and was
able to rescue most of my files. Learning Linux opened up a whole new
world of customization and configuration for me. When I first started, I found
it really interesting, intriguing, and even a little bit scary. Today, Linux
is the primary OS I use to work every day. We’ve learned a bunch of things
about operating systems and their histories. If all those different names
are making you feel like your human operating system is crashing, don’t
worry. You’ll pick it up as we go along. Now that we know we’re on the same page
when we say OS. In the next video, we’re going to get
our computer ready to work with Python.

What is the name of the command line tool commonly used to install, update, and remove external Python modules mentioned in the video?

pip

Awesome! pip is a cross-platform tool called a package manager used to install, update, or remove external Python modules.

They say that the best way to learn is
to practice, practice, practice, and then practice some more. So we
encourage you to practice the exercises in this course by running the scripts
on your computer. Doing this will help you hone your
programming skills as much as possible and can spark new ideas of things that
you could do through programming. As an IT specialist, it’s important to
know how to install and uninstall software in different operating
systems. To run Python locally, you either have
it installed on your computer. We’ll give you some pointers in the
next videos on how to do this for Python, but we also encourage you to
research how to manage your platform on your own. If you feel like you can use
a refresher on how to do some basic operations in your OS, like installing
packages or running commands, I recommend you take the course operating
systems and you, becoming a power-user. This course is taught by my colleague,
Cindy, and can really help you get up to speed on OSs. You’ll find a link to it in the next
reading. As we said in the last video, you can
use Python on all major operating systems. So it doesn’t matter if you’re
running Windows, macOS or Linux, you should be able to run Python locally on
your computer. It may even already be installed in
your system. To check whether you already have
Python installed on your computer, open a terminal or command prompt and
execute the Python command, passing –version as a parameter. Depending on what’s installed on your
computer, there’s a bunch of different things that can happen. It could say it
doesn’t recognize the command, which means you don’t have Python installed. It could return a version that starts
with the number two, which tells us that you have it installed but it’s Python

2. Or it could return a version that
   starts with the number three which means, you guessed it, you have Python
   3 installed. For this course, we’ll be using Python
3. Some of the things we’ll cover are the
   same on Python 2 and Python 3, but some can be pretty different. So make sure
   you have Python 3 installed. If running Python –version returns a
   version starting with two, try running Python 3 –version. Your computer could tell you that the
   command isn’t present which means you’ll have to install Python 3, or it can
   return a version that starts with number three which means you already have
   Python 3 installed. In the next few videos, we’ll talk
   about how to install and set up Python on each operating system if it’s not
   already there. After checking that you have Python 3
   installed, we’ll also learn about installing additional modules that
   aren’t part of the Python Standard Library. You’ll need these additional
   modules for some specific exercises in the course. What are these additional
   modules that we are talking about? In the intro to Python course we talked
   about the Python Standard Library. This comes as part of the Python
   installation and includes modules for the most common tasks you can do with
   Python. But there’s tons of other things you
   might want to do in your scripts and not all of them are in the standard
   library. This is where external modules come
   into play. We can use external modules for a bunch
   of tasks, like generating PDFs, starting web pages, creating compressed files,
   interacting with email, and a lot of other things. So how do we find what
   modules are available? When developers write a Python module
   that they think others might find useful, they publish it in PyPI, also
   known as the Python Package Index. We can browse as repository of Python
   modules to find the module we need. It includes thousands of projects which
   are classified by different categories, like topic, development studies, and
   intended audience. These external modules are generally
   managed with a command line tool called pip. This is a cross-platform tool so
   you can use it to install, update, and remove external modules on whichever
   operating system you’re running on your computer. Next up, we’re going to talk
   about installing main Python packages and installing external modules on
   Windows, macOS, and Linux. These videos are all optional. You can watch the ones most relevant to
   you. Or if you have Python already installed
   in your computer and you already know how to install external modules, feel
   free to skip ahead.

Windows computers don’t usually come
with Python preinstalled. To check if your computer already has
Python installed, run the command we talked about earlier, python–version. No, no Python here. Python isn’t installed on this
computer, so we’ll need to install it. There are a bunch of ways to do this on
Windows. We can download the installable package
from the official website and install that one, get it from the store if
we’re using Windows 10, or we can use a package management system called
Chocolatey to manage the installation. For this video, we’re just going to
install the package from the official website. But if you’d like to get a
taste of Chocolatey, you can check it out and download it on your own,
because who doesn’t like things a little extra chocolatey, right? To find the
installer, go to the official download page for windows. On this page, we can download the
executable installer for Python-3 64-bit architecture. Most computers today are
installed with 64-bit installations. So if you don’t know which one to pick,
pick that one, unless you know for sure that your computer is running a 32-bit
installation. Once the executable installer has
downloaded, we want to run it. This will install a new software on our
machine, so we need to run it as administrator user. Before clicking install now, make sure
you click on the add Python 3.7 to Path box. The path variable instructs the
operating system to look for executables and certain directories of our system
when running commands from the terminal. You want to have that box selected so
that python interpreter gets executed when we invoke it from the command
line. Great, it’s installing. It might take a moment. Now that it’s
done, we can test to see that it worked. We do this by opening a new PowerShell
and executing the same command as before. Nice. We now have an executable Python
interpreter that we can use for testing the scripts that we write. As we called
out, just because we’ve got Python-3 installed, it doesn’t mean that we have
the modules we might need for every possible script. Let’s say we’re tasked
with writing some automation that extracts information out of a website. To get the contents of a website from
Python, we can choose the request module, which is used for interacting
with web services. First, let’s check whether we have this
module already available. Darn. The interpreter tells us that the
module isn’t available. Let’s install it using Pip. To do that, we’ll call pip install
request from the command line, not the interpreter. We’ve now got the module installed. Let’s try to import it again from the
interpreter. Awesome, it worked. Just to double-check that, let’s try to
do something with this module. For example, we can use the get
function to get the contents of a website. So the get function process the website
and the response object now has its contents. We can do a bunch of things
with it. How about we check the length of the
response to text using the LEN function. Fantastic. Now that we have Python’s
setup on our Windows environment, we are ready to have some fun with it. Feel free to explore on your own and
try some things out. Up next, I’ll show you how to install
Python on macOS. You can check that one out or you can
skip ahead, it’s up to you.

By default, macOS ships with a version
of Python installed. But it’s Python 2, not Python 3. Let’s check this out. So while we have
Python installed on this computer, it’s only Python 2. I wonder if we have
Python 3 installed as a different command. Let’s check. So we don’t have Python 3 installed on
this computer. Let’s install it. There are two ways to
install Python on macOS. We can either download the installable
package from the official website and install that one, or we can use a
package management system called Homebrew to manage the installation. For this video, we’re going to install
the package from the official website but feel free to check out Homebrew on
your own. To install Python on macOS, go to the
official Python page at python.org. Now, we’ll click the Downloads menu. Because we’re running this on a macOS
computer, the website will automatically offer us a link to install Python 3 for
macOS. We’re now downloading the Python 3
executable installer for macOS. Once it’s done downloading, we’ll
execute it, go through the license in Terms of Use pages, and finally install
it. Depending on how your machine is set
up, you might need to enter your password to perform the actual
installation using administrator rights. Cool. Now we have an executable Python
3 interpreter that we can use for testing the scripts we’re going to
write. Let’s check that it’s working by
executing the same command as before. python3 –version. Great. Now we have Python 3 installed. Like we said earlier, just because we
have the interpreter installed, it doesn’t mean we have all the modules we
might need to execute every possible script. Let’s say we need to write an
automation script for handling events scheduling. To do that, we need to
manage a bunch of different dates and due date operations, like calculating
what date it will be in two weeks. To make our lives easier with all that
date manipulation, we could use the arrow module which makes it simple to
handle lots of different dates. First, let’s check whether we already
have this module available. Misfire. The interpreter says there’s
no arrow module available. To get that module, we’ll install it
using pip3 command. Notice that on Windows, the command was
called pip, on macOS, it’s called pip3. So to install the arrow module, we’ll
call pip3 install arrow, from the command line not the interpreter. Now that we got the module installed,
let’s try again to import it from the interpreter. Nice. Looks like that works. To check that we
can use it, let’s try to do something with this module. How about we use the
get function in the module to create a date object from a string. Great. Arrow parse a date from the
string in the format we told it to use. Now, the date object has that date. We can now operate with this object. For example, we can ask it to increment
it by six weeks using the shift method and then print it using the format
method. The format method that we just used can
receive a bunch of parameters. These parameters let us intuitively
format the string that we print. We’ve got our macOS environment ready
to go. Now we can use Python and start having
some fun with it. Feel free to explore on your own and
try some other things out. Up next, I’ll show you how to install
Python on Linux. You can check that one out or you can
skip ahead. It’s up to you.

If your computer runs Linux, chances are
extremely high that your distribution comes with Python already installed. You can check to see this by opening a
terminal and running a command like this. Okay. We have Python installed. But as we solve macOS, it’s Python 2. That’s an old version of Python, and we
want to use Python 3 instead. Let’s check whether we have that. Nice, Python 3 is already there. In some Linux distributions, the Python
command may already be using Python 3. In other distributions like this one,
you may need to use Python 3 as a command to access Python 3. In all of these courses, we’ll be using
Python 3, so make sure that you run the right command for your distribution. If for some reason your Linux
installation doesn’t have Python 3 installed, you’ll need to have it
installed using the package management system. The name of the tool to manage
local packages will depend on the distribution that you’re using. It’s called App on Debian Ubuntu and
Linux Mint, yum on Red Hat or CentOS, and DNF on Fedora. If you’re not sure what the right
command is for your distribution, it’s a good idea to do a little research to
find out. We’ve included a pointer to more
information in the next reading. While our computer here has Python 3
already installed, we still might not have all the modules needed to execute
all the scripts that’ll write, so we still need to learn how to add these
modules or systems, to make the most out of our Python installation. This example will be using an abundance
of computers, and the app tool to install Python packages. We’ll talk about two different ways of
installing packages. First, through the package management
system, and second, through Pip. As we said earlier, Linux’s
distributions usually have a package management system, and it’s a good idea
to use it to install additional software on your computer. Distributions usually
include separate packages for the different Python modules available. Say we’re tasked with writing some
Automation software that resizes images to fit a specific size. We can accomplish this by using the
Python imaging library known as PIL module. This includes a ton of image
manipulation functionality. To install this module on your Ubuntu
computer, we can use the app tool to install Python 3-pill Package. One of the advantages of using the
package management systems to install packages is that we can set the system
to automatically upgrade to newer versions. The package management system
makes sure that any dependencies are also installed so that the module is
ready to use. So let’s check out this module that we
just installed. The PIL module is very large, because
it includes tons of things that we can do for images. For this example, we
want to open our image and check its size and format. So we’ve imported the image sub-module
inside the PIL module. Now let’s open a file in our home
directory using the Open method. Great. We’ve opened the image. Now we can check its size and its
format attributes. Okay. The module is working properly. In this case, we wanted a module that
was provided by the package management system. So it was easy to just install
it using apps. In other cases, we may want to install
module versions that are newer than the ones provided in the distribution, or
they are not even present in this distribution at all. For cases like
these, we can use the same command we mentioned for Windows, and MacOS, the
pip command. For this scenario, we first need to
make sure that we have Python 3 version of pip which is called pip3, and is
provided by the Python 3 dash pip Package. We’ll do this by installing
the package to the package management system. This might take a minute. Now that we have pip3 installed, we can
solve problems using a wide range of external modules. For example, imagine
that you’re tasked with creating an automation that processes the error log
from a web service and then generates statistics on the amount of errors, or
when the most errors occur. To do this kind of Data Processing, we
could use our CUDA sounding module yet Pandas, which is widely used in the
Data Analysis field. All right, we have the Pandas module
installed. Let’s try it out with some sampled
numbers. Okay. We’ve imported the module. To state the obvious, since this is a
tool for Data Analysis, we’ll need some Data to use it. Let’s generate a few
sample lists, for the data associated with our website. How about a list of
the amount of visitors we’ve had in the last five days. Let’s add another list with the amount
of errors generated during the same five days. Now that we have these two lists, we
can use them to generate a Data Frame, which is the main structure used by the
pandas module to do data analysis. So we’ll generate a Data Frame with the
list, adding the days of the week as indexes like this. With this, we have a Data Frame that we
can use for operations. For example, we can print it in a
nicely formatted way. It’s always fun to print stuff, but
that’s not really Data Analysis. The Pandas module allows us to operate
on the numbers of each column. For example, we can calculate the mean,
or average, value of one of the columns using the mean method. Nice. We now know how to install Python
modules on Linux using either the package management system or pip. Are you ready to have some fun with it? Meme me over the next video where we’ll
talk about interpreted and compiled languages.

Congrats, you’ve finished
the first module of the course and you
made it through the first graded assessment
just like I knew you would. We’ve covered a lot of
ground in the past videos, so take a moment to
congratulate yourself. Seriously, you should feel
really good about this, not everyone sticks
with it like you have. Let’s quickly recap
what you’ve learned. We’ve talked about
some new concepts and some of you are probably
already a bit familiar with. We talked about what
operating systems are and why they’re relevant
to the world of IT, we went through how to install
Python on your machine, how to run Python scripts, how to create our own modules, and how to set up your
developer environment by choosing a code editor. We also learned more about Automation including
its benefits and its potential downsides
and check out some examples of when it’s a
good idea to use Automation. We looked through the
practical Automation example that we’ll keep expanding as we get further into the course. Last but not least, you did your first
Qwiklabs exercise of course where you put all this
new knowledge and practice, you fix the problem in a
Python script making it run correctly and you also wrote and use your own Python module. That’s amazing. Great job. Take a moment and give
yourself a pat on the back. Your hard work is
seriously paying off. Time to take a little
break if you need it and come back refreshed
for the next topic. In the next module, we’ll be talking
about managing files with Python. I’ll see you there.