ute2edriver (working name)

Drives a Lomiri app on a real Ubuntu Touch phone from the PC: taps, swipes, typing, screenshots and waiting on the app's log. It is for end-to-end checks and for the screenshots in docs and store listings, on the phone itself rather than an emulator. A person still signs a feature off on the phone; this does the repetitive part.

Tried on a Volla Phone 22 "Plinius", Ubuntu Touch 24.04, over USB.

Setup

• On the phone: developer mode on (Settings → About → Developer mode), the phone attached over USB and adb devices listing it. python3-evdev is in the system image. The default user (phablet) may write the input devices and /dev/uinput, so nothing runs as root.
• On the PC: uv and adb. uv brings the Python and Pillow (for cropping shots).
  • From a checkout: uv run ute2e ..., or from another folder (the app's, where its .ute2e.toml is) uv run --project ~/dev/ute2edriver ute2e ....
  • As a command on the PATH: uv tool install ~/dev/ute2edriver (add --editable to follow the checkout), then ute2e ....
• In the app's repository, a .ute2e.toml (see below). ute2e finds the nearest one from the working directory up.
• Keep the screen on and unlocked while it runs.

Commands

ute2e info                      touchscreen, screen size, grid unit, settings
ute2e launch [--fresh] | stop [APP_ID]  stop waits until it is gone; --fresh stops first
ute2e positions                 names the app logged, in app units
ute2e tap menu import           a logged name (or its last words: `tap import`)
ute2e tap 206 421               app units;  --px for screen pixels
ute2e swipe X1 Y1 X2 Y2 [--ms N]
ute2e type "Hello: world"       US layout, printable ASCII
ute2e erase 6 | key enter | hide-keyboard
ute2e shot out/name.png [--crop-top N]
ute2e wait 'myapp: ready' [--timeout S]
ute2e run flow.toml             a scripted run, see examples/

Every command holds a lock per phone (~/.cache/ute2edriver/), so two sessions never drive the same phone at once; run holds it for the whole flow. --ssh HOST uses ssh instead of adb, --serial picks an adb device.

The app's side: .ute2e.toml and position lines

app_id = "myapp.me_myapp_0.1"   # lomiri-app-list shows it while it runs
tag = "myapp: "                 # what the app's own log lines start with
origin = [0, 108]               # its window's top left on the screen, px
crop_top = 108                  # status bar height, cut off shots
# scale = 2.625                 # px per app unit: GRID_UNIT_PX / 8 by default

To be tapped by name, an element logs its centre in window coordinates when it is shown:

console.info("myapp: settings save " + Math.round(p.x) + " " + Math.round(p.y))
// where p = button.mapToItem(null, button.width / 2, button.height / 2)

The name is what comes between the tag and the first two numbers; words after them are ignored. The latest line for a name wins, read from the journal since the driver last launched the app (else the last half hour). Apps that log nothing can still be driven by coordinates.

Flows

A flow is a TOML list of steps, checked before any of them runs:

out = "shots"
[[step]]
launch = true
[[step]]
wait = "myapp: ready"     # regex, on lines since the step before it began
timeout = 20
[[step]]
tap = "menu import"
[[step]]
type = "https://example.org/a.zip"
[[step]]
shot = "import"           # shots/import.png

Steps: launch, stop, tap (name or [x, y] units), tap_px, swipe (ms), type, erase, key, hide_keyboard, wait (timeout), sleep, shot (crop_top). The log is marked before each action, so a wait after it cannot miss a line the action caused. A stop before the first launch starts from a fresh app (it waits until the old one is gone, see Stopping). examples/utstudybible/ has two that run against utStudyBible.

How it works, and what we found the hard way

• Taps are written straight to the phone's own touchscreen (the input device with direct multitouch, found by its capabilities). Lomiri ignores a uinput touchscreen.
• Typing and keys go through a uinput keyboard created per command; it needs about a second before the shell listens to it.
• Screenshots: mirscreencast aborts on some phones ("gralloc-mapper is missing"), so the driver presses volume up + down, Lomiri's screenshot keys, and collects the new file from ~/Pictures/Screenshots. Lomiri then shows a preview over everything; the driver taps its delete button ([device] screenshot_dismiss, default [60, 171] px on 1080×2400) and removes the file, so the phone is left as it was. If the preview is left up, every later tap lands on it.
• The on-screen keyboard ignores taps on its hide button; a swipe down on it hides it (hide_keyboard).
• Launching: lomiri-app-launch stays in the foreground and prints "Lost our connection with the registry"; the app starts anyway. It is run detached.
• Stopping: lomiri-app-stop does nothing for an app Lomiri no longer tracks (started before a shell restart, or by hand), and says nothing. So stop waits for the app's process (pidof, the id's middle part or process) to exit, then sends TERM, then KILL, and fails if it is still there. launch --fresh stops first, so a check never measures an app left over from an earlier run.
• Units: QML apps get GRID_UNIT_PX from Lomiri and a logical pixel is an eighth of it (21 px → 2.625 px per unit on the Plinius). The window starts below the status bar, hence origin.
• Screen off: taps do nothing and no screenshot is saved; the errors say so.

The phone side is one file, src/ute2edriver/agent.py, pushed to ~/.cache/ute2edriver/ whenever it changes.

Tests

uv run ruff check .
uv run ty check
uv run python -m unittest discover -s tests

Every function is annotated (ruff's ANN rules) and type-checked with ty. The tests cover the parts without a phone (positions, settings, flows, the commands sent). CI runs all three; the phone part is tried by hand.

Licence

GPL-3.0-or-later, see LICENSE.