This article is a continuation of the one previously published on Latency and Input Lag on Arcade in JAMMA

The previous article focused on the Recalbox RGB Jamma, this one will be more general about video games.

Introduction

We regularly hear about input lag and latency problems, particularly in retrogaming.

I'm sure you've come across it when dying in a game and blamed it, in infinite bad faith, on the developers who supposedly coded the game with their feet.

Everyone will feel it in their own way, but in general it's the most demanding gamers (professionals and heavy gamers), or people who want to recapture the original experience, who are most sensitive to it.

What is latency and input lag?

Simply put, latency is the time between pressing a button on your controller, and its visible action on screen, which can vary according to a number of factors.

For puzzle games or turn-based RPGs, it often has little impact on the gaming experience.

However, for faster games that require you to work your reflexes, such as shooters, platformers or fighting games, it can have a greater or lesser impact on the gaming experience.

By misuse of language, we often use the term Input lag when in fact it's only one component of overall latency.

Latency, or lag, can be broken down into three components:

Input lag: As its name suggests, input lag is the delay associated with input. It's the delay between the moment you press a button and the moment the signal reaches the console.

Process lag: This is the time it takes for the system/console to change the state of the game by applying the joystick event.

Video lag: Display lag is the time it takes for your screen to display the video signal supplied by the console.

Latency is usually measured in milliseconds (ms), but it can also be measured in frames.

What elements produce latency?

The controller

The controller (joystick, keyboard or mouse) you use can have an impact on latency.

If you have a no-name, low-cost, poorly designed controller, it won't help, far from it, because the electronics will be generic, inexpensive, and not necessarily designed for this use.

A good controller will have suitable electronics, designed for gaming and therefore reducing this input lag as much as possible.

Secondly, any wireless connection will physically add to input lag, to a greater or lesser extent depending on the type of connection used (mainly 2.4GHz or Bluetooth), and can add a few milliseconds of lag.

Infrared can also be found on older consoles, but is no longer used, as it has a number of drawbacks.

The hardware on which the game runs

In the case of hardware emulation via FPGA, the big advantage is that it emulates the original hardware at hardware level, and therefore without any conversion overlay to a third-party architecture, giving it a process lag equivalent to that of the original hardware.

Software emulation, on the other hand, is more variable: depending on the quality and implementation of the emulation, the complexity of the original hardware, and the computing power, you can get more or less close to the original hardware.

It should be borne in mind that the design of the original hardware remains largely non-public information, so we can only reverse-engineer it to deduce its behavior, and so emulation can hardly be perfect.

The primary aim of emulation, however, is to get as close as possible to the behavior of the original hardware.

The various adapter boxes.

This is not the case for everyone. If you use a box or cable, for example, to connect an original console to HDMI (known as a "scaler ”), or conversely if you use hardware to convert a digital signal to analog, depending on its design, this can add video lag. An HDMI splitter (to send the video signal to different screens) can also play on this video lag.

Similarly, if you're using a box to connect your old joysticks via USB, this can add input lag, depending on its design.

I think you'll have gathered that anything you add between you and your screen, in addition to the console, can potentially add latency.

If you really need this type of box or cable, especially for scalers and splitters, I suggest you watch the videos on the English-language youtube channel RetroRGB, which tries to test as many of these solutions as possible to identify the best ones.

Screens

The screen you use will also have an impact.

By the way, the youtube channel RetroRGB mentioned above has also carried out tests on this subject. As for CRT screens in general, there's no noticeable video lag. One exception is the latest generation of CRT screens, known as HD or 100/120Hz, which can add video lag during processing if these operating modes are activated.

Modern screens, on the other hand, more often cause problems. If you plug an old console directly into a modern screen (SCART, Composite, S-Video or Component), the screen will process the signal, transforming the analog signal into a digital one, and enlarging it to a definition that better matches the screen definition. But this processing takes time, and often affects the quality of the original image. It's often in this case that we think our vintage games were ugly, but it's actually partly the fault of the processing done by our screen (as well as the fact that CRT screens display the signal differently).

In addition to this, especially on modern TV screens, there can be a video lag added by “Cinema Mode” (or other) treatments, which has no impact when watching a movie, but is important for video games in general.

How to remedy it

For the joystick it's not going to be complicated, quite simply, choose a quality joystick, or a latency-free usb controller. This will guarantee good design quality and thus reduce input lag as much as possible.

Likewise for the joystick, choose wired** rather than wireless connections if you want to reduce input lag as much as possible.

For the screen, if possible, avoid any conversion made by your screen, and stick to the same type of signal throughout the chain. For example, connect an analog console (the consoles of your childhood) to a CRT screen that was made for it, and keep your flat screens only for connecting devices operating digitally via HDMI (Raspberry pi, PC, recent consoles, etc.). On these same flat screens, remember to activate the “Games” mode, often present, which will deactivate all potentially lag-adding processing.

Ideal for flat screens are gaming-oriented PC screens, which are designed to reduce these processing times as much as possible (generally between 1 and 5ms of added lag).

As for Recalbox on Raspberry pi, it can operate in both modes, in native digital mode and therefore connected to a modern screen (flat screens) via the HDMI socket, or in analog mode via the mini-jack socket (but with average signal quality) or via the Recalbox RGB Dual, designed specifically for this purpose.

If you can't or don't want to stick with a 100% analog channel for your old consoles, use a scaler made for video games. Avoid the cheap scalers, which are better suited to watching movies but not at all to video games. What's more, some of them are not compatible with RGB signals.

When it comes to emulation, the first thing to do is apply the latest emulator, core or system updates to take advantage of the latest optimizations and improvements. This is done simply by updating your recalbox.

In Retroarch (from which Recalbox inherits a number of cores and functionalities), there are options for reducing latency, by taking advantage of the computing power of our computers, and by calculating certain elements in advance, to adjust the delay and compensate for the delay that may be caused by the various elements in the chain.

How we work to reduce latency on Recalbox

As Recalbox is a software emulation solution, we are concerned by these latency problems and do our utmost to match the original hardware as closely as possible.

But before embarking on headlong development, we first need reference data, and to this end we've carried out a number of measurements.

Measurement methods: what already exists

Several methods exist for measuring latency. Depending on the method used, different components of latency can be measured.

To measure the input lag of a controller, you can use an oscilloscope to measure this lag precisely. More information on this method is available at this address. We used this method to check our input lag measurements.

It's possible to calculate display latency, using the 240p test suite, via its lag test (on the original console or in emulation), by visual comparison (using a camera), with two screens connected to the same source, it's possible to compare the difference in latency between the two screens. If this is done using a CRT screen as a reference (which has negligible display latency), it then becomes possible to calculate the display latency of a particular screen.

Other methods also exist, for example to measure display lag on a screen using Time Sleuth, which outputs a signal via HDMI and, with the help of a phototransistor, reads this signal and measures the display delay on the screen.

As these methods are not always easy to use, sometimes requiring expensive equipment, or are not comprehensive enough, we have also created our own tools, which we have compared with existing methods to check that they work properly.

Measurement methods: our developments

To better meet the need, we went so far as to develop our own tools to simplify these measurements.

First, we developed a software program to test input lag on a Raspberry Pi, which can be integrated and run on any distribution, including Recalbox.

All you need is a Raspberry Pi, a controller, some soldering to connect directly to the controller via the Raspberry Pi's GPIO port, and the program does the rest.

You can find the source code and documentation at this address.

Below are the input lag tools in operation, with the oscilloscope for comparison:

Secondly, we developed hardware: The Latency Bro. This enables us to take two types of measurements: input lag, and end-to-end latency (from button press to screen display), all autonomously.

It takes the form of a card plugged into the buttons on the controller/stick, with a USB port (for input lag measurement and power supply), and a phototransistor to detect a color change on the TV screen (when the controller is pressed). To measure latency, we ideally use the 240p test suite, which offers a test that switches from black to white when the joystick is pressed.

The Latency bro then displays the latency measured over one or more tests (with automatic averaging).

Here's an overview of the latency bro in operation:

Results

We carried out measurements on various controllers/handlers, and also compared original hardware and emulation under Recalbox.

Input Lag

As you can see from the measurements taken, the controller used can add almost up to one frame of delay (for those tested), which is very little in itself, but you need to look at the whole chain right up to the display to see the full impact on the user.

Latency

For latency, we compared two consoles, and their equivalent in emulation on Recalbox: the Playstation 1, and the Super NES, in 60Hz on all hardware.

To be exhaustive on the emulation side, we tested different cores with different parameters.

The Playstation 1 controller used was connected to Recalbox via a “NoName” USB adapter.

On the hardware side for emulation, we used a Raspberry Pi 4 with a Recalbox RGB Dual, these measurements were taken in 2022, so we don't yet have measurements on the Raspberry Pi 5. We'll be taking measurements in the future on this more recently arrived card.

We'd also like to thank @FFVIMan who carried out the lag test for the Super NES hardware.

Following these tests, we can see that the average latency, which we used as a reference, for these two consoles is 41 ms for the PS1 and 30 ms for the Super NES.

In the various tests carried out, you'll also see that we've worked to find the parameters that best optimize the emulation to get closer to the reference for each console. As emulation quality evolves over time, these settings are bound to evolve in order to optimize them.

To simplify the task and give you back the experience of the original hardware, we've pre-configured this in the form of an option to be activated in the menus dedicated to Recalbox RGB Dual, under the heading “Reduce Latency”.

In fact, these options will no longer be limited to *RGB Dual/JAMMA users from recalbox version 10 onwards, and you'll be able to activate them easily even via HDMI.

Conclusion

Emulation is often accused of being the source of latency. However, with the right configuration, the latency added by the emulator becomes imperceptible. Add to this low-latency controllers and a suitable TV/monitor, and Recalbox offers you a solution that comes as close as possible to the original experience. New and even more effective options are coming in version 10!

In any case, I hope this article has enlightened you on the subject of latency.