MicroPython 'gc' Module

Contents

• Introduction
• What is Garbage Collection?
• Module gc Functions
• Module gc Examples
• MicroPython Tutorial Links

Introduction

MicroPython is a 'slim' version of the ever popular Python programming language. With its much smaller footprint it is an effective and easily learnt language for programming microcontrollers.

The BBC micro:bit was originally designed and mass produced as an educational device. However many enthusiasts have discovered this interesting little device and it is increasingly becoming explored by microcontroller hobbyists.

Often the micro:bit's MicroPython port is a subset of the MicroPython standard. However in the case of the gc module it seems to be completely implemented.

This module might not one that most developers will ever use. However it does have its uses when optimising microcontroller performance speed e.g. when attempting to precisely time a process, such as a GPIO pin state change, down to microseconds.The functions in the module are very easy to use once the concept of garbage collection is understood.

After a brief introduction to the garbage collection service this tutorial will provide an alphabetical list of the seven gc functions followed by a brief description of each. Finally a MicroPython program, tested on a micro:bit, will be presented to demonstrates each function's usage.

What is Garbage Collection?

As a program runs it will usually need to store data that has been assigned to variables. Many variables during the lifetime of the program will firstly become in scope when declared and assign values to allocated memory.

Many of these variable e.g. local variables used in functions and methods will eventually lose scope e.g. when the execution point moves beyond the function or method. The data is dereferenced with its allocated memory released for reuse. If this release (deallocation) process didn't occur the system might eventually run out of RAM.

The process of allocating and deallocating memory is performed by the garbage collection service. Some languages e.g. C/C++ require input from the developer to ensure garbage collection (memory deallocation) occurs. However MicroPython uses an automatic memory management scheme 'behind the scenes'.

While in most occasions the MicroPython programmer doesn't need to even be aware of this automated garbage collection service, the module gc does provide manual intervention control - if required.

With micro:bit's MicroPython, most data is stored in a large preassigned area of memory called the heap. A 32-bit pointer is created separately that points (holds the physical memory address) to the variable's value on the heap.

The only exception to the heap being used for variable data storage is if the data to be stored will fit in the 32-bit pointer i.e. small integers, booleans and values such as Nothing. Everything else goes to the heap.

Garbage collection is automatically initiated when there is a demand to allocate heap memory and there is insufficient memory available to satisfy the demand.

When the garbage collection service needs to do a memory cleanup it will begin by marking 'live' objects that are still referenced. This is known as the mark phase. Finally during the sweep phase all unmarked objects are reclaimed, freeing up memory for reuse.

Module gc Functions

The following is an alphabetical listing of the seven gc module functions with links to their descriptions below.:

Before any of these functions can be used an import statement is required:

import gc
          

Example
import gc
print('Disabling garbage collection')
gc.disable()
print('garbage collection enabled?',gc.isenabled() )
print('Enabling garbage collection')
gc.enable()
print('garbage collection enabled?',gc.isenabled() )

Output:
Disabling garbage collection
garbage collection enabled? False
Enabling garbage collection
garbage collection enabled? True
          

Module gc Examples

The first example calls a function that builds a list of 2,00 large integers. This will almost exhaust the heap memory. The functions gc.mem_free() and gc.mem_alloc() are used to report on the RAM available and used up respectively on the heap before and after building the large memory sapping list.

Finally the function gc.collect() is called to manually initiate a garbage collection.

Example 1
# Demonstrates usage of functions
# from the gc module.

import gc
import urandom # Generates random numbers

def BigData():
    # Generates 2,000 large random
    # intgers and inserts into a list.
    biglist = list()
    for i in range(2000):
        bigint = urandom.getrandbits(32)
        biglist.append(bigint)

print('Stage 1:')
print("Program's initial heap RAM")
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
print()

# Use up most of the available heap.
print('Stage 2:')
BigData()
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
print()

# Do this again.
# MicroPython will do a garbage
# collection to release memory.
print('Stage 3:')
BigData()
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
print()

# Do a manual garbage collection.
print('Stage 4:')
print('Initiate manual garbage collection:')
gc.collect()        
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
          

Output:

Stage 1:
Program's initial heap RAM
Heap memory free: 63248
Heap memory used: 1264

Stage 2:
Heap memory free: 6176
Heap memory used: 58336

Stage 3:
Heap memory free: 7776
Heap memory used: 56736

Stage 4:
Initiate manual garbage collection:
Heap memory free: 63152
Heap memory used: 1360
          

Stage 1: The heap memory that is still free and heap memory that has been allocated is reported :

As is to be expected most of the heap is unused.

Stage 2: The memory consuming functionBigData() is called. The heap memory that is still free and heap memory that has been allocated is reported :

The BigData() function has used up most of the available heap memory. When execution passes out of the function the 2,000 large integers that it has created are still on the heap but have been dereferenced i.e. no longer available to the program. At this stage garbage collection has not yet occurred.

Stage 3: The BigData() function is called again. This time the heap will quickly run out of free memory. The MicroPython runtime detects this and immediately runs a garbage collection otherwise an out of memory exception (error) would be thrown. This doesn't happen which proves the garbage collection to remove the unmarked objects still on the heap from the function call in Stage 2 has been successful.

The heap memory is still just about exhausted but this time from now deallocated data created during the most recent call to the function.

Stage 4: This time a manual initiation of garbage collection is initiated.

It's obvious from the high number of free bytes now available on the heap that the garbage collection has successfully freed up the unmarked objects from the second call to the BigData() function.

This example modifies Example 1 to prevent a garbage collection from occurring before the second call to the BigData() function in Stage 3. This has the inevitable result of causing the program to run out of heap memory and terminating with a fatal error condition.

The function gc.disable() is to disable garbage collection. The function gc.isenabled() is called to confirm that garbage collection really is disabled.

CAUTION: This program can leave the micro:bit in a 'hung' state with the REPL unresponsive. If this occurs immediately flash another program to restore 'normal service'. If in doubt, DON'T RUN THIS PROGRAM.

Example 2
# Demonstrates usage of functions
# from the gc module.

import gc
import urandom # Generates random numbers

def BigData():
    # Generates 2,000 large random
    # intgers and inserts into a list.
    biglist = list()
    for i in range(2000):
        bigint = urandom.getrandbits(32)
        biglist.append(bigint)

print('Stage 1:')
print("Program's initial heap RAM")
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
print()

# Use up most of the available heap.
print('Stage 2:')
BigData()
print('Heap memory free:', gc.mem_free())    
print('Heap memory used:', gc.mem_alloc())
print()

# Do this again but block the
# garbage collection from running.
# This will cause an out of memory
# exception to occur.
print('Stage 3:')
# Turn garbage collection off.
gc.disable()
# Check that garbage collection is off.
print('Garbage collection enabled?', gc.isenabled())
BigData()
          

Output:

Stage 1:
Program's initial heap RAM
Heap memory free: 63552
Heap memory used: 960

Stage 2:
Heap memory free: 7344
Heap memory used: 57168

Stage 3:
Garbage collection enabled? False
MemoryError: 
FATAL: uncaught NLR 200149b0
          

As expected the micro:bit halts with a fatal error when it runs out of memory.

Sometimes it may be necessary to restrict garbage collection before starting a very time critical operation. If the garbage collection is to be turned off for whatever reason this must be done with caustion and it is usually good practice to firstly force a manual garbage collection. And don't forget to turn it back on!

The following example sorts the 'out of memory' fatal termination that occurred in Example 2  by performing a manual garbage collection before the garbage collection is disabled.

Example 3
# Demonstrates usage of functions
# from the gc module.

import gc
import urandom # Generates random numbers

def BigData():
    # Generates 2,000 large random
    # intgers and inserts into a list.
    biglist = list()
    for i in range(2000):
        bigint = urandom.getrandbits(32)
        biglist.append(bigint)

print('STAGE 1:')
print("Program's initial heap RAM")
print('Heap memory free:', gc.mem_free())
print('Heap memory used:', gc.mem_alloc())
print()

# Use up most of the available heap.
print('STAGE 2:')
BigData()
print('Heap memory free:', gc.mem_free())
print('Heap memory used:', gc.mem_alloc())
print()

# This time a manual garbage collection
# will be forced before garbage collection
# is turned off.
print('STAGE 3:')
# Force garbage collection
gc.collect()
print('Forcing garbage collection...')
print('Heap memory free:', gc.mem_free())
print('Heap memory used:', gc.mem_alloc())
# Turn garbage collection off.
# Check that it is off.
print('Turning garbage collection off...')
gc.disable()
print('Garbage collection enabled?', gc.isenabled())
print("\nRunning 'BigData()' again...")
BigData()
print('Heap memory free:', gc.mem_free())
print('Heap memory used:', gc.mem_alloc())
# Turn garbage collection back on.
print('\nTurning garbage collection back on...')
gc.enable()
# Check that it is on.
print('Garbage collection enabled?', gc.isenabled())
          

Output:

STAGE 1:
Program's initial heap RAM
Heap memory free: 62912
Heap memory used: 1600

STAGE 2:
Heap memory free: 7632
Heap memory used: 56880

STAGE 3:
Forcing garbage collection...
Heap memory free: 62848
Heap memory used: 1664
Turning garbage collection off...
Garbage collection enabled? False

Running 'BigData()' again...
Heap memory free: 6288
Heap memory used: 58224

Turning garbage collection back on...
Garbage collection enabled? True
          

No memory issues this time because of the manual garbage collection performed before turning off the automated garbage collection!