MicroPython : Working with Files

Contents

• Preamble
• Introduction
• Opening a File
• Reading from a File
  • The read() Method
  • The readline() Method
• Writing to a File
• Closing Files
  • File Operations - ‘try … finally’ Block
  • File Operations - ‘with’ Clause
• MicroPython Tutorial Links

Preamble

MicroPython is a “slim” version of Python specifically designed with a small footprint to efficiently run on memory constrained microcontrollers.

MicroPython doesn't implement anywhere near the complete complement of Python file handling functionality. However the subset that is implemented is somewhat useful and might be all that is required, with one exception^[1], for most microcontroller applications.

Important Note: This document specifically describes the MicroPython file operations available for the file system on the BBC micro:bit. This file system is flat i.e. does not allow a hierarchical folder structure and is capacity limited to about 30KB.

All examples in this article are original and have been tested on a micro:bit for correctness using the Mu Editor.

Introduction

Computers are often used to process vast amounts of information. Whether it is in numeric, text or binary form it often becomes important to have a capable file system to store this data both pre and post processing.

Microcontrollers on the other hand are more often used to solve real-time problems of a less computational nature. It could be simply to read a sensor and write a value to a single line display or maybe use it to control some process eg operating a valve or an electrically driven pump to maintain the level of liquid in a tank.

Thus the microcontroller has far less use for a sophisticated file system than its larger computer cousin. However the microcontroller's ability to create, read, write and delete files can have some practical uses.

A microcontroller may store startup parameters in a file in non-volatile memory that is read in each time a particular program is loaded and executed. This could be likened to a Microsoft Window application's ini file.

A microcontroller might be operating as a data logger. In this case there may be several sensors connected to ports on the microcontroller which is tasked with periodical reads of these sensors. The values as they are collected will need to be written to log files for later downloading and analysis.

In this scenario it is more likely that an external microSD board will be interfaced with the microcontroller. These boards are inexpensive, quite easy to setup and have the advantage that the MicroSD card can be physically removed to transfer log files to computers, etc.

Opening a File

Like most things in MicroPython, a file is an object with associated properties and methods. The open() function, acting as a class constructor, sets up a file for reading or writing operations and thens returns a file object.

The syntax is simple:

open(<filename>, <mode>)

Example:
  f = open("test.txt", "rt")

          

MicroPython only supports read (default) "r" and write "w" modes. Specifically, MicroPython does not support the append mode of the Python language standard.

File data can be either text (default) "t" or binary "b". So the allowable list of mode options are "r", "w", "t", "b", "rt", "rb", "wt", "wb".

The default is "rt" if no mode is specified.

In text mode all data including numbers are converted to strings. In binary mode the data is pure binary e.g. a graphics file.

Reading from a File

Before a file can be read from it must be opened for reading using the open() function. If the file cannot be found then MicroPython will raise an OSError exception.

# Open a text file for reading
open('test.txt')
open('test.txt', 'r')  
open('test.txt', 't')
open('test.txt', 'rt')

# Open a binary file for reading
open('test.bin', 'b')  
open('test.bin', 'rb')

          

MicroPython supports two methods for reading from a file:

1. read()
2. readline()

<file>.read([<size]>)

Parameters:>
  file is the file object from the open() function call.
  size (optional) is the number of characters to read.
      The default is -1 meaning the entire file is read.

Examples
  buffer = f.read() #read the entire file
  buffer = f.read(5) #read 5 characters

            

ABCDEFGHIJ
KLMNOPQRST
UVWXYZ

            

# Open a text file for reading
f = open("test.txt", "rt")

# Read two characters
s = f.read(2)
print('First two characters:', s)

# Read the next three characters
s = f.read(3)
print('Next three characters:', s)

# Read the remainder of the file
s = f.read()
print('Remainder of the file:\n', s)

# Attempt to read beyond the end of the file
s = f.read()
print('Number of characters remaining:', len(s))

            

First two characters: AB
Next three characters: CDE
Remainder of the file:
 FGHIJ
KLMNOPQRST
UVWXYZ

Number of characters remaining: 0
            

<file>.readline([<size]>)

Parameters:>
  file: The file object from the open()
        function call.

  size (optional) is the number of characters to read.
      The default is -1 meaning characters are read
      from the  file till either a new line 
      character ('\n') or the end of the file
      is reached (whichever occurs first).

Examples:
    buffer = f.readline() # Read one line
    buffer = f.readline(5) # Read 5 characters

# Demonstrate the use of readline()

# Open a text file for reading
f = open("test.txt", "rt")

# Read the first two characters of the first line
s = f.readline(2)
print('First two characters of Line 1:', s)

# Read the next three characters of the first line
s = f.readline(3)
print('Next three characters of Line 1:', s)

# Read the remainder of the first line
s = f.readline()
print('Remainder of Line 1:', s)

# Read the second line
s = f.readline()
print('Line 2:', s)

# Read the third line
s = f.readline()
print('Line 3:', s)

First two characters of Line 1: AB
Next three characters of Line 1: CDE
Remainder of Line 1: FGHIJ

Line 2: KLMNOPQRST

Line 3: UVWXYZ

            

Writing to a File

Before a file can be written to, it must be opened for writing using the open() function. If the file doesn't exist on the microcontroller's filesystem than it will be created. If the file already exists then MicroPython will overwrite it.

Sadly, MicroPython does not implement a file append mode.

# Open a text file for writing
open('test.txt', 'w')  
open('test.txt', 'wt')

# Open a binary file for writing
open('test.bin', 'wb')  

          

MicroPython provides only one method, write(), for writing data to a file. The syntax is very simple.

<file>.write(<data>)

Parameters:>
    file: The file object from
          the open() function call.

    data: Text or binary bytes
          to be written to the file.

          

The program is responsible for inserting new line ('\n') characters at the end of lines when writing to text files. MicroPython does not automatically do this.

The following example opens a new text file for writing. Some text is written to the file and it is closed. The use of the close() method and its rationale is explained next in Closing Files.

The file is reopened, this time for reading. The file is read and the text is displayed.

Two additional functions supported by MicroPython are the writable() and name() file methods.

The method writable() returns True if the file can be written to; otherwise it returns false.

The method name() returns the name of the file associated with the file object as a string. This method works with files opened for reading or writing. Both of these file methods are also demonstrated in the next example.

A file can be deleted using the remove() method^[2] from the MicroPython standard os standard library. If the file doesn't exist an OSError exception will occur.

# Demonstrate write(), writable()
# and name() methods.

# Provides access to remove() function.
import os

# Text to write to the file
sentence1 = "Mary had a little lamb"
sentence2 = "It's fleece was as white as snow"

# File name
filename = "spam.txt"

# Open a file for writing text.
# If the file already exists
# in the micro:bit filesystem
# it will be overwritten.
f = open(filename, "wt")

# Check that the file is ready for writing
print('The file', f.name(), 'is ready for writing:',
       f.writable(), '\n')

# Write the text to the file and close it.
f.write(sentence1 + '\n')
f.write(sentence2 + '\n')
f.close()

# Open the file for reading
f = open(filename)
print(f.read())
f.close()

# Delete the file
print('Deleting file:', filename)
os.remove(filename)

The file spam.txt is ready for writing: True 

Mary had a little lamb
It's fleece was as white as snow

Deleting file: spam.txt

Closing Files

The Python interpreter running a program on a computer makes calls to the underlying operating system (OS) to perform file operations. This causes the OS to allocate resources that become unavailable to other applications running in a multitasking environment.

Furthermore, Python's default algorithm for writing to a file is to firstly write the data to a buffer in memory. Python makes a request to the OS to flush the buffer out to the physical file location when it becomes full.

Even though Python does a good job of cleaning up resources automatically when no longer required, it is considered good programming practice (and indeed necessary with some language) to explicitly 'close' a file when all operations have been completed on it. This will cause the OS to flush any write buffers to physical storage and release any allocated resources.

The situation with MicroPython is somewhat different. Generally microcontrollers will be running a single program.

In the case of the micro:bit, when it receives a reset signal from either powering up or the reset button is pushed, the MicroPython interpreter will fire up and look for main.py.

There are no other applications competing for resources. When a program requests file services MicroPython must provide it as there is no underlying operating system to call.

MicroPython does a good job of cleaning up file resources under normal circumstances. The issue of possible data loss and file corruption does arise though if an unexpected error event (exception) occurs. MicroPython provides two mechanisms to solve this:

1. File Operations - try … finally Block
2. File Operations - with Clause

# Demonstrates the use of exception handling
# to flush the write buffer and clean up
# file resources.

# Five random integers (100..200) are
# written to a text file. The file is
# reopened and the integers are read.
# The sum is calculated and reported.

# Provides random numbers.
import random
# Provides access to remove() method.
import os

# Open the file for writing.
# If it already exits then it
# will be overwritten.
filename='numbers.txt'
f = open(filename, 'w')
try:
    # Write five random integers
    # to the file.
    for i in range(1, 6):
        f.write(str(random.randint(100, 200)))
finally:
    # Flush the buffer to the physical file.
    f.close()

# Open the file for reading
# It is known that the integers
# are all three characters in
# length (between 100 and 200).
sum = 0
print('Random integers (100-200):')
f = open(filename)
try:
    # Read first integer
    s = f.read(3)
    while s != '':
        # Read remaining integers
        print(s, end=' ')
        sum += int(s)
        s = f.read(3)
    print('\nSum is:', sum)
finally:
    # Close the file and delete it.
    f.close()
    print('Deleting file:', filename)
    os.remove(filename)

Random integers (100-200):
188 100 156 138 103 
Sum is: 685
Deleting file: numbers.txt

with expression as <target_var>:
    do_something(<target_var>)

Example:
    with open("hello.txt", mode="w") as file:
        file.write("Hello, World!")

# Demonstrates the use of the 'with'
# statement to flush the write buffer
# and clean up file resources.

# Five random integers (100..200) are
# written to a text file. The file is
# reopened and the integers are read.
# The sum is calculated and reported.

# Provides random numbers.
import random
# Provides access to remove() method.
import os

filename='numbers.txt'

# Open the file for writing.
# If it already exits then it
# will be overwritten.
with open(filename, 'w') as f:
    for i in range(1, 6):
        # Write five random
        # integers to the file.
        f.write(str(random.randint(100, 200)))

sum = 0
print('Random integers (100-200):')

# Open the file for reading
# It is known that the integers
# are all three characters in
# length (between 100 and 200).
with open(filename) as f:
    # Read first integer
    s = f.read(3)
    while s != '':
        # Read remaining integers
        print(s, end=' ')
        sum += int(s)
        s = f.read(3)
    print('\nSum is:', sum)
    # Delete the file
    print('Deleting file:', filename)
    os.remove(filename)

Random integers (100-200):
124 156 199 120 179 
Sum is: 778
Deleting file: numbers.txt