As part of our short course on Python for Physics and Astronomy we consider how users interact with their computing environment. A programming language such as Python provides tools to build code that computes scientific models, captures data, sorts it and analyzes it largely without operator action. In effect, once you have written the program, you point it at the data or task it is to do, and wait for it to return new science to you. This is the command line, or batch, model of computing and is at the core of large data science today. Indeed, from your handheld devices to supercomputers, the work that is done is for the most part autonomous. We have seen how Python has built-in components to accept input from the command line, the operating system, the computer that is hosting the program, and the Internet or cloud. What about the other side, the user's perspective on computing?
As an end user, would you prefer to move a mouse or tap a screen in order to select a file, or to type in the path and file name? What if you had to make operational decisions based on graphical output, or changing real world environments as data are collected? In modern computing, most of us interact with the machine and software through a graphical user interface or GUI.
Command Line Interfacing and Access to the Operating System
In a unix-like enviroment (Linux or MacOS), the command line is an accessible and often preferred way to instruct a program on what to do. A typical program, as we've seen, might start like this example to interpolate a data file and plot the result:
import sys import numpy as np from scipy.interpolate import UnivariateSpline import matplotlib.pyplot as plt
sfactorflag = True
if len(sys.argv) == 1: print " " print "Usage: interpolate_data.py indata.dat outdata.dat nout [sfactor]" print " " sys.exit("Interpolate data with a univariate spline\n") elif len(sys.argv) == 4: infile = sys.argv outfile = sys.argv nout = int(sys.argv) sfactorflag = False elif len(sys.argv) == 5: infile = sys.argv outfile = sys.argv nout = int(sys.argv) sfactor = float(sys.argv) else: print " " print "Usage: interpolate_data.py indata.dat outdata.dat nout [sfactor]" print " " sys.exit("Interpolate data with a univariate spline\n")
It uses "sys" to parse the command line arguments into text and numbers that control what the program will do. Because its first line directs the system to use the python interpreter, if the program is marked as executable to the user it will run as a single command followed by arguments. In this case it would be something like
interpolate_data.py indata.dat outdata.dat nout sfactor
where indata.dat is a text-based data file of x,y pairs, one pair per line, outdata.dat is the interpolated file, nout is the number of points to be interpolated, and sfactor is an optional floating point smoothing factor. When you run this it will read the files, do the interpolation without further interaction, and (as written) plot a result as well as write out a data file. The rest of the code is
# Take x,y coordinates from a plain text file # Open the file with data infp = open(infile, 'r') # Read all the lines into a list intext = infp.readlines() # Split data text and parse into x,y values # Create empty lists xdata =  ydata =  i = 0 for line in intext: try: # Treat the case of a plain text comma separated entry entry = line.strip().split(",") # Get the x,y values for these fields xval = float(entry) yval = float(entry) xdata.append(xval) ydata.append(yval) i = i + 1 except: try: # Treat the case of a plane text blank space separated entry entry = line.strip().split() xval = float(entry) yval = float(entry) xdata.append(xval) ydata.append(yval) i = i + 1 except: pass # How many points found? nin = i if nin < 1: sys.exit('No objects found in %s' % (infile,))
# Import data into a np arrays x = np.array(xdata) y = np.array(ydata)
# Function to interpolate the data with a univariate cubic spline if sfactorflag: f_interpolated = UnivariateSpline(x, y, k=3, s=sfactor) else: f_interpolated = UnivariateSpline(x, y, k=3)
# Values of x for sampling inside the boundaries of the original data x_interpolated = np.linspace(x.min(),x.max(), nout) # New values of y for these sample points y_interpolated = f_interpolated(x_interpolated)
# Create an plot with labeled axes plt.figure().canvas.set_window_title(infile) plt.xlabel('X') plt.ylabel('Y') plt.title('Interpolation') plt.plot(x, y, color='red', linestyle='None', marker='.', markersize=10., label='Data') plt.plot(x_interpolated, y_interpolated, color='blue', linestyle='-', marker='None', label='Interpolated', linewidth=1.5) plt.legend() plt.minorticks_on() plt.show()
# Open the output file outfp = open(outfile, 'w') # Write the interpolated data for i in range(nout): outline = "%f %f\n" % (x[i],y[i]) outfp.write(outline) # Close the output file outfp.close() # Exit gracefully exit()
Aftet the fitting is done the program runs pyplot to display the results. The interactive window it opens and manages is a GUI, but it has been set up by the command line code. Of course there are many variations on command line interfacing, and the one shown here with coded argument parsing is perhaps the simplest and would serve as a template for most applications. Python offers other ways to manage the command line too. The os module is useful to have access to the operating system from within a Python routine. Some examples are
os.chdir(path) changes the current working directory (CWD) to a new one os.getcdw() returns the CWD os.getenv(varname) returns the value of the environment variable varname
and there are many more, providing within the Python program many of the command line operating system tools available on the system. Here's an example of how that might be used in a program that processes many files in a directory:
# Process images in a directory tree
import os import sys import fnmatch import string import subprocess import pyfits
if len(sys.argv) != 2: print " " sys.exit("Usage: process_fits.py directory\n")
toplevel = sys.argv
# Search for files with this extension pattern = '*.fits'
for dirname, dirnames, filenames in os.walk(toplevel): for filename in fnmatch.filter(filenames, pattern): fullfilename = os.path.join(dirname, filename) try: # Open a fits image file hdulist = pyfits.open(fullfilename) except IOError: print 'Error opening ', fullfilename break
# Do the work on the files here ... # You can call a separate system process outside of Python this way darkfile = 'dark.fits' infilename = filename outfilename = os.path.splitext(os.path.basename(infilename))+'_d.fits' subprocess.call(["/usr/local/bin/fits_dark.py", infilename, darkfile, outfilename])
Here we used the os module's routines to walk through a directory tree, parse filenames, and then perform another operation on those files that is a separate command line Python program. Command line tools used to leverage the operating system's built-in functions can be very powerful, and take hours out of actually running a program on a large database.
Graphical User Interfaces Along with Plotting
First, read the comprehensive section on Tkinter to see how that code works. This section is an update of an older description of how to use Tk with matplotlib.
First, we tell the operating system to use Python, and we comment the code with a simple description of what it does.
#!/usr/bin/python """ Interactively plot data using matplotlib pyplot within a Tk root window """
We import sys and numpy modules and use numpy as "np" in the usual way.
import sys import numpy as np
We import matplotlib and call it mpl for a short name. We tell mpl to use Tk by default.
import matplotlib as mpl mpl.use('TkAgg')
We import some backend things that may (or may not) be needed to get plotting to work in our own window.
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg from matplotlib.backend_bases import key_press_handler
We import Figure which does does the plotting for us
from matplotlib.figure import Figure
Importing Tk is handled differently in Python 2 and Python 3. We also get askopenfilename which we will use to get a file using a GUI.
if sys.version_info < 3:
import Tkinter as Tk from tkFileDialog import askopenfilename
import tkinter as Tk from tkFileDialog import askopenfilename
Define a function that will load a new file into the dataspace once its name is known.
def load_file(newfile): # Take x,y coordinates from a plain text file # Open the file with data
global x global y global nin try: infp = open(newfile, 'r') except: return(0) # Read all the lines into a list intext = infp.readlines()
# Split data text and parse into x,y values
# Create empty lists
xdata =  ydata =  i = 0
for line in intext:
try: # Treat the case of a plain text comma separated entry entry = line.strip().split(",")
# Get the x,y values for these fields xval = float(entry) yval = float(entry) xdata.append(xval) ydata.append(yval) i = i + 1
except: try: # Treat the case of a plane text blank space separated entry
entry = line.strip().split()
xval = float(entry) yval = float(entry) xdata.append(xval) ydata.append(yval) i = i + 1 except: pass # How many points found?
nin = i if nin < 1: sys.exit('No objects found in %s\n' % (infile,))
# Import data into a np arrays
x = np.array(xdata) y = np.array(ydata)
This program optionally will take a filename on the command line. We define a flag to tell us later if there was one provided when the program started.
fileflag = True
Parse the command line and set the fileflag according to the user's wishes.
if len(sys.argv) == 1: fileflag = False elif len(sys.argv) == 2: fileflag = True infile = sys.argv else: print " " print "Usage: pyplot_data.py [indata.dat]" print " " sys.exit("\nUse pyplot to display a data file\n")
If there was no file provided, bring up a GUI to allow the user to select a file.
if not fileflag: root = Tk.Tk() infile = askopenfilename() root.quit() root.destroy()
After the file selection has been done, we need to remove traces of the window so it will not clutter the desktop or cause errors later.
npts = load_file(infile)
if npts <= 0: print "Could not find data in %s \n" % (infile,) exit()
The data are now loaded and ready to plot, so the rest is GUI content.
Create the root/toplevel window with title
root = Tk.Tk() root.wm_title("PyPlot")
Set up the figure for the plot. Here we use a 7x5 aspect ratio at 200 DPI. On the screen his is nominally a 7x5 inch display, but depends on the monitor. Try 16x9 with 100 DPI for a larger monitor. Add a title to the plot and label the axes.
f = Figure(figsize=(7,5), dpi=200) a = f.add_subplot(111) p, = a.plot(x,y) a.set_title(infile) a.set_xlabel('X') a.set_ylabel('Y')
Create tk.DrawingArea with mpl figure
canvas = FigureCanvasTkAgg(f, master=root) canvas.show() canvas.get_tk_widget().pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)
Optionally add the default mpl plotting toolbar (comment out to disable)
toolbar = NavigationToolbar2TkAgg( canvas, root ) toolbar.update()
Pack the canvas to make things fit nicely.
canvas._tkcanvas.pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)
Add the keypress event handler that is built into mpl. This may not be necessary, but it seems wise.
def on_key_event(event): print('You pressed %s'%event.key) key_press_handler(event, canvas, toolbar)
Add a Quit button as an example of what you might include on this canvas.
# Stop mainloop root.quit() # Prevent error running on Windows OS root.destroy() button = Tk.Button(master=root, text='Quit', command=mpl_quit) button.pack(side=Tk.RIGHT)
Add a File button to change the file once the plot is displayed
def mpl_file(): infile = askopenfilename() npts = load_file(infile) if npts > 0: p.set_xdata(x) p.set_ydata(y) canvas.show() return
button = Tk.Button(master=root, text='File', command=mpl_file) button.pack(side=Tk.LEFT)
Now we are ready to run the loop interact with the graphics.
The program runs waiting for user input. When you move the mouse, those events are trapped by mpl and used to update the cursor readout, or to activate the buttons on the mpl toolbar. The File button will allow selection of a new file to plot. When you click your Exit button the root window will be removed and the program will exit.
python -m CGIHTTPServer 8000 1>/dev/null 2>/dev/null & echo "Use localhost:8000" echo
By using port 8000 the server is distinct from the one on port 80 used for web applications. The site would appear by putting
in a Google Chrome or Mozilla Firefox browser window running on the same user account on the same machine. Note the redirects for stdio and stderr to /dev/null keeps output from appearing in the console. The server may be killed by identifying its process ID in Linux with the command
ps -e | grep python
kill -s 9 pid
where "pid" is the ID number found in the first line. Alternatively, if it is the only python process running you may kill it with
For programmers, however, this utility allows development and debugging of web software without the need for a large server.