PyMOTW: readline

By Doug Hellmann
November 30, 2008

readline – Interface to the GNU readline library

Purpose:Provides an interface to the GNU readline library for interacting with the user at a command prompt.
Python Version:1.4 and later

The readline module can be used to enhance interactive command line programs to make them easier to use. It is primarily used to provide command line text completion, or “tab completion”.

Note

Because readline interacts with the console content, printing debug messages makes it difficult to see what it happening in the sample code versus what readline is doing for free. The examples below use the logging module to write debug information to a separate file. The log output is shown with each example.

Configuring

There are two ways to configure the underlying readline library, using a configuration file or the parse_and_bind() function. Configuration options include the keybinding to invoke completion, editing modes (vi or emacs), and many other values. Refer to the GNU readline library documentation for details.

The easiest way to enable tab-completion is through a call to parse_and_bind(). Other options can be set at the same time. This example changes the default editing controls to use “vi” mode instead of the default of “emacs”. To edit the line, press ESC then normal vi navigation keys.

import readline

readline.parse_and_bind('tab: complete')
readline.parse_and_bind('set editing-mode vi')

while True:
line = raw_input('Prompt ("stop" to quit): ')
if line == 'stop':
break
print 'ENTERED: "%s"' % line


The same configuration can be stored as instructions in a file read by the library with a single call. If myreadline.rc contains:


# Turn on tab completion
tab: complete

# Use vi editing mode instead of emacs
set editing-mode vi


the file can be read with read_init_file():


import readline

readline.read_init_file('myreadline.rc')

while True:
line = raw_input('Prompt ("stop" to quit): ')
if line == 'stop':
break
print 'ENTERED: "%s"' % line




Completing Text


As an example of how to build command line completion, we can look at a program that has a built-in set of possible commands and uses tab-completion when the user is entering instructions.


import readline
import logging

LOG_FILENAME = '/tmp/completer.log'
logging.basicConfig(filename=LOG_FILENAME,
level=logging.DEBUG,
)

class SimpleCompleter(object):

def __init__(self, options):
self.options = sorted(options)
return

def complete(self, text, state):
response = None
if state == 0:
# This is the first time for this text, so build a match list.
if text:
self.matches = [s
for s in self.options
if s and s.startswith(text)]
logging.debug('%s matches: %s', repr(text), self.matches)
else:
self.matches = self.options[:]
logging.debug('(empty input) matches: %s', self.matches)

# Return the state'th item from the match list,
# if we have that many.
try:
response = self.matches[state]
except IndexError:
response = None
logging.debug('complete(%s, %s) => %s',
repr(text), state, repr(response))
return response

def input_loop():
line = ''
while line != 'stop':
line = raw_input('Prompt ("stop" to quit): ')
print 'Dispatch %s' % line

# Register our completer function
readline.set_completer(SimpleCompleter(['start', 'stop', 'list', 'print']).complete)

# Use the tab key for completion
readline.parse_and_bind('tab: complete')

# Prompt the user for text
input_loop()


The input_loop() function simply reads one line after another until the input value is "stop". A more sophisticated program could actually parse the input line and run the command.


The SimpleCompleter class keeps a list of “options” that are candidates for auto-completion. The complete() method for an instance is designed to be registered with readline as the source of completions. The arguments are a “text” string to complete and a “state” value, indicating how many times the function has been called with the same text. The function is called repeatedly with the state incremented each time. It should return a string if there is a candidate for that state value or None if there are no more candidates. The implementation of complete() here looks for a set of matches when state is 0, and then returns all of the candidate matches one at a time on subsequent calls.


When run, the initial output looks something like this:


$ python readline_completer.py
Prompt ("stop" to quit):

If you press TAB twice, a list of options are printed.


$ python readline_completer.py
Prompt ("stop" to quit):
list print start stop
Prompt ("stop" to quit):

The log file shows that complete() was called with two separate sequences of state values.


$ tail -f /tmp/completer.log
DEBUG:root:(empty input) matches: ['list', 'print', 'start', 'stop']
DEBUG:root:complete('', 0) => 'list'
DEBUG:root:complete('', 1) => 'print'
DEBUG:root:complete('', 2) => 'start'
DEBUG:root:complete('', 3) => 'stop'
DEBUG:root:complete('', 4) => None
DEBUG:root:(empty input) matches: ['list', 'print', 'start', 'stop']
DEBUG:root:complete('', 0) => 'list'
DEBUG:root:complete('', 1) => 'print'
DEBUG:root:complete('', 2) => 'start'
DEBUG:root:complete('', 3) => 'stop'
DEBUG:root:complete('', 4) => None

The first sequence is from the first TAB key-press. The completion algorithm asks for all candidates but does not expand the empty input line. Then on the second TAB, the list of candidates is recalculated so it can be printed for the user.


If next we type “l” and press TAB again, the screen shows:


Prompt ("stop" to quit): list

and the log reflects the different arguments to complete():


DEBUG:root:'l' matches: ['list']
DEBUG:root:complete('l', 0) => 'list'
DEBUG:root:complete('l', 1) => None

Pressing RETURN now causes raw_input() to return the value, and the while loop cycles.


Dispatch list
Prompt ("stop" to quit):

There are two possible completions for a command beginning with “s”. Typing “s”, then pressing TAB finds that “start” and “stop” are candidates, but only partially completes the text on the screen by adding a “t”.


The log file shows:


DEBUG:root:'s' matches: ['start', 'stop']
DEBUG:root:complete('s', 0) => 'start'
DEBUG:root:complete('s', 1) => 'stop'
DEBUG:root:complete('s', 2) => None

and the screen:


Prompt ("stop" to quit): st


Warning


If your completer function raises an exception, it is ignored silently and
readline assumes there are no matching completions.





Accessing the Completion Buffer


The completion algorithm above is simplistic because it only looks at the text argument passed to the function. It is also possible to use functions in the readline module to manipulate the text of the input buffer.


import readline
import logging

LOG_FILENAME = '/tmp/completer.log'
logging.basicConfig(filename=LOG_FILENAME,
level=logging.DEBUG,
)

class BufferAwareCompleter(object):

def __init__(self, options):
self.options = options
self.current_candidates = []
return

def complete(self, text, state):
response = None
if state == 0:
# This is the first time for this text, so build a match list.

origline = readline.get_line_buffer()
begin = readline.get_begidx()
end = readline.get_endidx()
being_completed = origline[begin:end]
words = origline.split()

logging.debug('origline=%s', repr(origline))
logging.debug('begin=%s', begin)
logging.debug('end=%s', end)
logging.debug('being_completed=%s', being_completed)
logging.debug('words=%s', words)

if not words:
self.current_candidates = sorted(self.options.keys())
else:
try:
if begin == 0:
# first word
candidates = self.options.keys()
else:
# later word
first = words[0]
candidates = self.options[first]

if being_completed:
# match options with portion of input
# being completed
self.current_candidates = [ w for w in candidates
if w.startswith(being_completed) ]
else:
# matching empty string so use all candidates
self.current_candidates = candidates

logging.debug('candidates=%s', self.current_candidates)

except (KeyError, IndexError), err:
logging.error('completion error: %s', err)
self.current_candidates = []

try:
response = self.current_candidates[state]
except IndexError:
response = None
logging.debug('complete(%s, %s) => %s', repr(text), state, response)
return response

def input_loop():
line = ''
while line != 'stop':
line = raw_input('Prompt ("stop" to quit): ')
print 'Dispatch %s' % line

# Register our completer function
readline.set_completer(BufferAwareCompleter(
{'list':['files', 'directories'],
'print':['byname', 'bysize'],
'stop':[],
}).complete)

# Use the tab key for completion
readline.parse_and_bind('tab: complete')

# Prompt the user for text
input_loop()


In this example, commands with sub-options are are being completed. The complete() method needs to look at the position of the completion within the input buffer to determine whether it is part of the first word or a later word. If the target is the first word, the keys of the options dictionary are used as candidates. If it is not the first word, then the first word is used to find candidates from the options dictionary.


There are three top-level commands, two of which have subcommands:



  • list

    • files

    • directories



  • print

    • byname

    • bysize



  • stop


Following the same sequence of actions as before, pressing TAB twice gives us the three top-level commands:


$ python readline_buffer.py
Prompt ("stop" to quit):
list print stop
Prompt ("stop" to quit):

and in the log:


DEBUG:root:origline=''
DEBUG:root:begin=0
DEBUG:root:end=0
DEBUG:root:being_completed=
DEBUG:root:words=[]
DEBUG:root:complete('', 0) => list
DEBUG:root:complete('', 1) => print
DEBUG:root:complete('', 2) => stop
DEBUG:root:complete('', 3) => None
DEBUG:root:origline=''
DEBUG:root:begin=0
DEBUG:root:end=0
DEBUG:root:being_completed=
DEBUG:root:words=[]
DEBUG:root:complete('', 0) => list
DEBUG:root:complete('', 1) => print
DEBUG:root:complete('', 2) => stop
DEBUG:root:complete('', 3) => None

If the first word is “list ” (with a space after the word), the candidates for completion are different:


Prompt ("stop" to quit): list
directories files

The log shows that the text being completed is not the full line, but the portion after


DEBUG:root:origline='list '
DEBUG:root:begin=5
DEBUG:root:end=5
DEBUG:root:being_completed=
DEBUG:root:words=['list']
DEBUG:root:candidates=['files', 'directories']
DEBUG:root:complete('', 0) => files
DEBUG:root:complete('', 1) => directories
DEBUG:root:complete('', 2) => None
DEBUG:root:origline='list '
DEBUG:root:begin=5
DEBUG:root:end=5
DEBUG:root:being_completed=
DEBUG:root:words=['list']
DEBUG:root:candidates=['files', 'directories']
DEBUG:root:complete('', 0) => files
DEBUG:root:complete('', 1) => directories
DEBUG:root:complete('', 2) => None



Input History


readline tracks the input history automatically. There are two different sets of functions for working with the history. The history for the current session can be accessed with get_current_history_length() and get_history_item(). That same history can be saved to a file to be reloaded later using write_history_file() and read_history_file(). By default the entire history is saved but the maximum length of the file can be set with set_history_length(). A length of -1 means no limit.


import readline
import logging
import os

LOG_FILENAME = '/tmp/completer.log'
HISTORY_FILENAME = '/tmp/completer.hist'

logging.basicConfig(filename=LOG_FILENAME,
level=logging.DEBUG,
)

def get_history_items():
return [ readline.get_history_item(i)
for i in xrange(1, readline.get_current_history_length() + 1)
]

class HistoryCompleter(object):

def __init__(self):
self.matches = []
return

def complete(self, text, state):
response = None
if state == 0:
history_values = get_history_items()
logging.debug('history: %s', history_values)
if text:
self.matches = sorted(h
for h in history_values
if h and h.startswith(text))
else:
self.matches = []
logging.debug('matches: %s', self.matches)
try:
response = self.matches[state]
except IndexError:
response = None
logging.debug('complete(%s, %s) => %s',
repr(text), state, repr(response))
return response

def input_loop():
if os.path.exists(HISTORY_FILENAME):
readline.read_history_file(HISTORY_FILENAME)
print 'Max history file length:', readline.get_history_length()
print 'Startup history:', get_history_items()
try:
while True:
line = raw_input('Prompt ("stop" to quit): ')
if line == 'stop':
break
if line:
print 'Adding "%s" to the history' % line
finally:
print 'Final history:', get_history_items()
readline.write_history_file(HISTORY_FILENAME)

# Register our completer function
readline.set_completer(HistoryCompleter().complete)

# Use the tab key for completion
readline.parse_and_bind('tab: complete')

# Prompt the user for text
input_loop()


The HistoryCompleter remembers everything you type and uses those values when completing subsequent inputs.


$ python readline_history.py
Max history file length: -1
Startup history: []
Prompt ("stop" to quit): foo
Adding "foo" to the history
Prompt ("stop" to quit): bar
Adding "bar" to the history
Prompt ("stop" to quit): blah
Adding "blah" to the history
Prompt ("stop" to quit): b
bar blah
Prompt ("stop" to quit): b
Prompt ("stop" to quit): stop
Final history: ['foo', 'bar', 'blah', 'stop']

The log shows this output when the “b” is followed by two TABs.


DEBUG:root:history: ['foo', 'bar', 'blah']
DEBUG:root:matches: ['bar', 'blah']
DEBUG:root:complete('b', 0) => 'bar'
DEBUG:root:complete('b', 1) => 'blah'
DEBUG:root:complete('b', 2) => None
DEBUG:root:history: ['foo', 'bar', 'blah']
DEBUG:root:matches: ['bar', 'blah']
DEBUG:root:complete('b', 0) => 'bar'
DEBUG:root:complete('b', 1) => 'blah'
DEBUG:root:complete('b', 2) => None

When the script is run the second time, all of the history is read from the file.


$ python readline_history.py
Max history file length: -1
Startup history: ['foo', 'bar', 'blah', 'stop']
Prompt ("stop" to quit):

There are functions for removing individual history items and clearing the entire history, as well.




Hooks


There are several hooks available for triggering actions as part of the interaction sequence. The startup hook is invoked immediately before printing the prompt, and the pre-input hook is run after the prompt but before reading text from the user.


import readline

def startup_hook():
readline.insert_text('from startup_hook')

def pre_input_hook():
readline.insert_text(' from pre_input_hook')
readline.redisplay()

readline.set_startup_hook(startup_hook)
readline.set_pre_input_hook(pre_input_hook)
readline.parse_and_bind('tab: complete')

while True:
line = raw_input('Prompt ("stop" to quit): ')
if line == 'stop':
break
print 'ENTERED: "%s"' % line


Either hook is a potentially good place to use insert_text() to modify the input buffer.


$ python readline_hooks.py
Prompt ("stop" to quit): from startup_hook from pre_input_hook

If the buffer is modified inside the pre-input hook, you need to call redisplay() to update the screen.



See also



readline

The standard library documentation for this module.

GNU readline

Documentation for the GNU readline library.

readline init file format

The initialization and configuration file format.

effbot: The readline module

Effbot’s guide to the readline module.

cmd

The cmd module uses readline extensively to implement
tab-completion in the command interface. Some of the examples here
were adapted from the code in cmd.

rlcompleter

rlcompleter uses readline to add tab-completion to the interactive
Python interpreter.



PyMOTW Home


You might also be interested in:

News Topics

Recommended for You

Got a Question?