Beyond colour gamuts: Novel metrics for the reproduction of photoreceptor signals

Colour gamuts describe the chromaticity reproduction capabilities of a display, i.e. its ability to reproduce the relative cone excitations from real-world radiance spectra. While the cones dominate “canonical” visual function (i.e. perception of colour, space, and motion) under photopic light levels, they are not the only photoreceptors in the human retina. Rods and melanopsin-containing intrinsically photosensitive retinal ganglion cells (ipRGCs) also respond to light and contribute to both visual and non-visual light responses, including circadian rhythms, sleep-wake control, mood, pupil size, and alertness. Three-primary display technologies, with their focus on reproducing colour, are not designed to reproduce the rod and melanopsin excitations. Moreover, conventional display metrics used to characterize three-primary displays fail to describe the display’s ability (or inability) to reproduce rod and melanopsin excitations, and thus do not capture the display’s ability to reproduce the full human physiological response to light. In this paper, three novel physiologically relevant metrics are proposed for quantifying the reproduction and distortion of the photoreceptor signals by visual displays. A novel equal-luminance photoreceptor excitation diagram is proposed, extending the well-known MacLeod-Boynton chromaticity diagram, to allow visualizations of the ﬁve-dimensional photoreceptor signal space in a three-dimensional projection.


Introduction
As the world becomes ever more reliant on display technology, display manufacturers continue to aspire towards creating hyper-realistic displays that can reproduce scenes as they appear in the real-world.One goal of display technology is to induce the full human physiological response to the light in a displayed scene that occurs in the corresponding real-world scene.The full human physiological response to light is described in a five-dimensional sensor space where the dimensions correspond to the excitation of the three cone classes, the rods, and the melanopsin-containing intrinsically photosensitive retinal ganglion cells (ipRGCs).
There are currently no metrics to characterise a display's ability (or inability) to reproduce the full five photoreceptor response to light.If future developments in display technology are geared towards better reproduction of the full human physiological response to light, including the reproduction of rod and melanopsin excitations, then metrics that characterize the full physiological response to light are necessary to guide such development.Further, visualization methods are needed to illustrate the five-dimensional photoreceptor space.In this paper, we propose three novel physiologically relevant metrics that can be used to characterize a display's ability to reproduce all five photoreceptor signals.Additionally, we propose an equal-luminance photoreceptor excitation diagram, as an extension of the well-known MacLeod-Boynton chromaticity diagram [1], to allow for visualization of the full physiological space.

The human physiological response to light
To understand the need to reproduce and quantify the reproduction of all five photoreceptor signals in order to characterize a display, it is worth first unpacking the physiology behind the human light response.The first step in the human response to light is the absorption of photons by a photoreceptor in the retina.Vision is an obvious example of a light-driven mechanism: photons are absorbed by the three different types of cones, the long-wavelength sensitive (L), medium-wavelength sensitive (M), and short-wavelength sensitive (S) cones.Comparisons by retinal neurons with centre-surround receptive fields support pre-processing of spatial variation.Comparisons between cone types with different spectral sensitivity allow chromatic discrimination.
The cones are not the only photoreceptors in the retina: they share the role of light sensation with the rods and the melanopsin-containing ipRGCs.The visual response to light, therefore, is more nuanced than the cone-driven model of photopic vision.Rods are well-known to contribute to visual function at low-light levels.While rods have been traditionally thought to saturate in bright light conditions [2,3] the exact light level at which rods begin to saturate, or the variability in this level between observers is not well-defined, and recent evidence suggests a rod-response is still persistent even in highly photopic conditions [4,5].Even in the classical rod-saturation model, most office lighting levels would fall comfortably within the mesopic range, meaning rods are likely to contribute to much of our daily visual function [6,7].There is also evidence that ipRGC responses may reach the visual cortex, and thus that melanopsin signals may also be contributing to some visual functions [8][9][10][11][12][13].
Beyond vision, light is well-known to be one of the primary drivers of the entrainment of the internal circadian clock to the external light-dark cycle and exposure to bright light in the evening and at night can suppress the production of melatonin, sometimes colloquially referred to as the "sleep hormone" [14][15][16].Further, the spectral composition of light, not just the integrated radiance, is important for driving these circadian features, with the spectral sensitivity of these non-visual effects of light being consistent with the spectral sensitivity of melanopsin [17,18].Thus melanopsin excitation is often attributed as being the primary driver of these non-visual circadian effects of light.ipRGCs receive additional synaptic inputs from rods and cones [19], and therefore rods and cones may also be contributing to the light driven control of circadian-rhythms.
ipRGCs are known to work alongside the cones to drive other non-visual responses to light.For example, pupil size change is thought to be mediated by both melanopsin and cone signals [10,[20][21][22][23][24][25].L and M-cone signals combine with melanopsin signals to drive pupil size while S-cone and melanopsin influence pupil size in an opponent manner [26,27].
ipRGCs also play an important role in driving the light induced maintenance of alertness [28][29][30].Additionally, light exposure can impact mood, with light therapy being a treatment option for Seasonal Affective Disorders (SADs), and other mental health problems [16].In sum, ipRGCs underlie key influences of light on physiology and behaviour.
While there is sufficient evidence implicating roles of melanopsin, rods, and cones in various visual and non-visual pathways, there is still a lot unknown about how the post-receptoral pathways combine the signals from different photoreceptors for various functions.An exhaustive review of the evidence for and a complete discussion of the post-receptoral pathways that may combine rod, cone, and melanopsin signals to drive visual and non-visual responses to light is beyond the scope of this paper.Instead, this paper focuses on the implications of the complex picture of visual and non-visual responses to light for image reproduction systems.

The reproduction of light by displays
Most modern displays, which make use of wide colour gamuts, high-resolution screens, and high-dynamic range are capable of reproducing the spatial and chromatic characteristics of real-world scenes.Consider reproducing a single chromatic sample, which occupies the entire field of view (such as the radiance spectrum shown in Fig. 1(b)), on a display (with primaries such as those in Fig. 1(c)) as a simple uniform colour patch.To match the colour on the display to the colour from the real-world radiance spectrum one can simply calculate the relative intensity vector, = [ , , ], of the primaries of the display, ( ), ( ), and ( ), needed to generate a spectrum on the display (Fig. 1(d)) that reproduces the L, M, and S responses, = [ , , and ], from the real-world spectrum (for chromaticities that fall within the display gamut), as in Eq. 1 (note, the Stockman-Sharpe 10 • cone fundamentals, ( ), ( ), ( ), as in the CIE S026 standard and depicted in Fig. 1(a), are used to calculate the L, M, and S responses [31][32][33][34]).As the L, M, and S responses from both the display and the real-world spectrum will be equal, as shown in Fig. 1(e), the colour of the display and of the illuminated surface is predicted to match.
Traditional image reproduction systems use only three primaries to generate a visual scene, matching the L, M, and S cone signals from the real-world scene to be replicated with the reproduced scene.In doing so, the rod and melanopsin signals, and , are uncontrolled in the reproduced scene.The reproduced rod and melanopsin signals, * and * , are determined by the spectral output, ( ) of the display, determined by the relative intensity vector [ , , ] set to reproduce the cone responses as described in Eq. 1 (note, the CIE S026 standard spectral sensitivity functions of rods, ( ), and melanopsin, ( ), are used here and depicted in Fig. 1(a) [17,34,35]).The distorted rod and melanopsin signals are described in Eq. 2, and an example of how the signals are distorted when an example spectrum is displayed on the hypothetical three-primary display depicted in Fig. 1(c) is shown in Fig. 1(e).
This could have major consequences for displays: lack of explicit control of the rod and melanopsin signals may induce inadvertent visual effects in the reproduced scene and inadvertent regulation of the non-visual responses to light, including circadian rhythms, pupil size control, alertness, and mood [36].
The inability of traditional displays to control melanopsin and rod excitation poses a problem as the world continues to move online and become heavily reliant on display technology in every day life.As people may use computers, TVs, and phones well into the evening and night, the unnatural melanopsin signals arising from displays at such late hours of the day when melanopsin signals from natural, real-world light at such times would be lower, could lead to large disruptions in sleep quality, circadian cycle, and mood in the general population.While many newer displays are trying to combat this problem with a "night-shift" mode, which simply shifts the white balance of the display towards longer wavelength, reducing the amount of short-wavelength light (a) The spectral sensitivity functions of the cones, rods, and melanopsin (as in the CIE S026 standard) [17,31,32,34,35].(b) An example radiance spectrum from a real-world reflectance surface viewed under a real-world illuminant.(c) Example primaries of a hypothetical three-primary display.(d) The spectral output of the hypothetical three-primary display (c) that reproduces the same cone excitations as the real-world radiance spectrum (b).(e) The cone, rod, and melanopsin excitations from the real-world radiance spectrum ((b), with black outline) and from the hypothetical three-primary display's spectral output ((d), grey outline).(f) Example primaries of a hypothetical five-primary display.(g) The spectral output of the hypothetical five-primary display (f) that reproduces the same photoreceptor excitations as the real-world radiance spectrum (b).(h) The photoreceptor excitations from the real-world spectrum (b, black outline) and the hypothetical five-primary display's spectral output (g, grey outline).One can see that while the rod and melanopsin excitations from the real-world spectrum and hypothetical three-primary display output do not match even though the cone excitations do, all five photoreceptor signals match between the real-world spectrum and hypothetical five-primary display.exposure of the evening, doing so alters the chromaticity and thus the visual appearance of the display.Current "night-shift" modes do not directly tackle the problem of controlling melanopsin signals, and instead try to suppress melanopsin activity by a general short-wavelength reduction, which in turn affects the cone signals.If the goal is to minimise circadian disruption but to maintain illuminance, it is useful to know that dimming will have a much stronger effect than colour tuning [37].In addition to minimizing inappropriate melanopsin activation, one can also think of instances where the active control of the melanopic irradiance could be beneficial, such as tailored light exposure to mitigate circadian disruption in shift work or in inter-timezone travel.
3D displays have long been afflicted by the inability to reproduce the focus cues of a real image correctly, and therefore struggle to generate a 3D percept without associated visual discomfort [38].3D displays may also suffer as a result of uncontrolled melanopsin responses.Melanopsin has been consistently shown to drive the pupillary light reflex.In traditional three-primary 3D displays, the melanopsin signal is not controlled, which could reduce depth of field to increase cue-conflict issues, though whether or not the pupil size change driven by melanopsin would be of a sufficient magnitude to have any effect remains unclear and an interesting area of future empirical testing.

Towards more accurate physiologically relevant reproduction of light on displays
Both hardware and software developments may help improve a display's ability to more accurately reproduce the five photoreceptor signals.Five-primary displays offer an exciting solution to control melanopsin and rod signals directly, without distorting the cone responses [22,39].With five primaries one is able to directly control the signal of all five photoreceptors, as long as the signals are within gamut, as described in Eq. 3, and illustrated by an example in Fig. 1 For three-primary displays, it may be possible to design a more optimal chromaticity transformation which allows for some distortion in the cone responses to introduce less distortion in the melanopsin and rod signals, though extensive research would be needed to demonstrate that any such algorithm produced a scene that had acceptable chromaticities and visual properties.cone fundamentals, though still only offers a quantification of the cone signals, not rods and melanopsin [33].There is therefore a need to look beyond traditional and physiologically relevant colour spaces which only offer a three-dimensional snippet of the five-dimensional physiological space, in search of a better-suited, physiologically-relevant visualization and accompanying metrics to quantify display reproduction.
Three novel, physiologically relevant metrics are proposed in this paper as a means of quantifying the reproduction and distortion of photoreceptor signals on displays.The first proposed metric, the photoreceptor signal reproduction metric (PSRM), quantifies the ability of a display to accurately reproduce a set of correlated quintuplets of photoreceptor signals.The second metric, the photoreceptor signal distortion metric (PSDM), quantifies any distortions introduced into any of the photoreceptor signals on the display compared to the desired signal.The final metric, the photoreceptor correlation distortion metric (PCDM), quantifies any distortions introduced to the correlations between the five photoreceptor signals from the desired signals to the displayed signals.In addition to the novel metrics, an equal-luminance photoreceptor excitation diagram, an extension of the MacLeod-Boynton chromaticity diagram [1], is proposed to allow for a physiologically relevant visualization of the real-world and reproduced photoreceptor signals.

Methods: Introducing three novel metrics and an equal-luminance photoreceptor excitation diagram
One of challenges for designing metrics to quantify reproduction of rod and melanopsin signals, is deciding what reference dataset to use.For chromaticity reproduction, the goal has often been to achieve a gamut that reaches the spectrum locus in order to display all physically possible visible spectra, but in reality displays are still a long way from achieving that.The sRGB, Adobe RGB, and Rec.709 gamuts have become popular standard gamuts to act as a realistic reference against which to compare chromaticity reproduction in displays [41][42][43].There is no equivalent reference gamut for rods and melanopsin.Instead, one must define an appropriate reference dataset with which to compare rod and melanopsin signal reproduction to.One option, which is presented in this paper, is to consider the plausible spectra encountered in the real-world as the reference dataset.In the real-world, much of our visual input originates from objects that do not themselves emit light, but that are visible by virtue of the light they reflect.For matte surfaces this process is modelled by a surface being illuminated by a particular illuminant, and the multiplication of the surface reflectance and illuminant spectrum gives the resultant radiance spectrum which the observer sees.Illuminant spectra are available through specular reflection from glossy surfaces, or from direct viewing.An ideal reference dataset might perhaps contain all possible real-world surfaces and illuminants, weighted by the likelihood of an observer to interact with each surface under each illuminant.Such a dataset does not yet exist [44].However, there are datasets of real-world surfaces and real-world illuminants, from which a reference dataset of radiance spectra can be derived, that captures the variety of surfaces and illuminants that one may encounter in the real-world.Such a dataset is used here as a reference against which to quantify the reproduction and distortion of the reproduced photoreceptor quintuplet signals of a display.
The real-world reference frame used here compromises 39,699 different radiant spectra, derived from 99 real-world reflectance samples under 401 different illuminants [45,46].The term real-world spectra here refers to both natural and artificial illuminants and surfaces.The illuminant database contains the daylight series, black-body radiators, narrowband and broadband flourescent lamps, LEDs, Tungsten Filament bulbs, and other miscellaneous illuminants.The reflectance database contains samples from nature, skin, textiles, paints, plastics, prints, and colour systems.Full details of the illuminant and reflectance datasets are detailed elsewhere [45,46], and the appropriateness of this dataset is further evaluated in the Discussion.
The radiant spectra, ( ), are calculated from all possible combinations of a reflectance spectrum, ( ), and an illuminant spectrum, ( ), as described in Eq. 4.
As the focus of this paper is on the reproduction of the relative photoreceptor quintuplets, not the reproduction of the absolute signals, the illuminant spectra were normalized to have unit area.

Metric I: Photoreceptor Signal Reproduction Metric (PSRM)
The first proposed metric quantifies how many of the real-world photoreceptor signal quintuplets can be accurately reproduced on the display which is useful for comparing the extent of the five-dimensional gamut across different displays.The metric is termed the photoreceptor signal reproduction metric, or PSRM.This can be thought of as a five-dimensional extension of the traditional colour gamut, i.e. the number of photoreceptor quintuplets that can be captured by the display (to within a predefined tolerance).A PSRM of 100% would indicate the display is able to reproduce all of the real-world photoreceptor quintuplets from the reference dataset, while a PSRM of zero would indicate the display is unable to reproduce any of the real-world photoreceptor quintuplets.Simply put, the PSRM quantifies the percentage of all real-world photoreceptor signal quintuplets, [ , , , , ], that are able to be reproduced on the display, [ * , * , * , * , * ], defined as in Eq. 5.
The reproduced photoreceptor quintuplets are only considered plausible and thus counted in the above sum if all of the primaries of the display are set to non-negative, and non-zero values for the signal to be reproduced.All possible 39,699 real-world quintuplets are counted in the denominator, to quantify how many real-world quintuplets are accurately reproduced.The reproduced photoreceptor quintuplets, [ * , * , * , * , * ], are only counted if they are equal to the the real-world quintuplet that is being reproduced, [ , , , , ].In practice, a certain level of tolerance is necessary.Here, each photoreceptor signal quintuplet is considered to be reproduced on the display, i.e. * = , if the reproduced signal, * , is within 1% of error of the real-world signal, (the error metric is defined in Eq. 6).* − < where here = 0.01 ∈ [ , , , , ] In this metric, one does not explicitly consider the distortion introduced by three-primary displays, though the effects of the distortion on the rod and melanopsin signals will simply reduce the PSRM of the display.

Metric II: Photoreceptor Signal Distortion Metric (PSDM)
The second metric, the photoreceptor signal distortion metric, or PSDM, explicitly quantifies the distortion introduced into the uncontrolled photoreceptor signals.This metric describes the degree of failure to reproduce each of the real-world photoreceptor signals, which is important to characterize a display's inability to reproduce the real-world signals and the impact the introduced distortions may have on the observer.For instance, a display that introduces a slight distortion in the melanopsin signals may be preferable to a display that introduces large distortions in the melanopsin signals when the PSRM of the two displays is equivalent.The PSDM is defined as the contrast between the reproduced signal and the real-world signal for all realisable spectra (i.e.all spectra that fall within gamut) as in Eq. 7. The overall PSDM is defined as the mean of the absolute value of the PSDM across all spectra and all photoreceptors.
One can see that the error metric in Metric I, the PSRM, which gives the tolerance for whether photoreceptor quintuplets are considered to be reproduced is the PSDM defined above.By design and definition, the colours reproduced by the display do not change the cone signals (PSDM = PSDM = PSDM = 0).It would be possible to define a chromaticity transformation that fixed the melanopsin signal instead of the M cone signal for example, thus meaning the melanopsin distortion would be zero by definition, and introducing distortions to the M cone signal, or devising a transformation that balances the distortion across all five photoreceptor classes.

Metric III: Photoreceptor Correlation Distortion Metric (PCDM)
It is not just the photoreceptor signals, but rather the correlations between the photoreceptor signals that provides information about the radiance spectrum that is falling on the retina.Spectral information from multiple photoreceptors depends on the correlations between them.For example, L and M cone signals are highly correlated and the distinction along the red-green colour opponent axis depends on the decorrelation between the L and M cone signals.In real-world spectra there are distinct correlations between the different photoreceptor excitations, as shown in Fig. 2. Fig. 2(a) shows the excitations for each photoreceptor pair from the 39,699 real-world spectra, while Fig. 2(b) shows the Pearson correlation coefficient describing the correlations between each pair of photoreceptor signals.In particular the L and M cone excitations are highly correlated, as are the rod and melanopsin excitations.If one or more of the photoreceptor signals are distorted when reproduced on a display, then the correlations between that photoreceptor signal and the remaining photoreceptors may be distorted.Hence, the third and final metric, the photoreceptor correlation distortion metric, or PCDM, is defined.It is worth highlighting the distinction between the PSDM and the PCDM in terms of the information they provide about the nature of the distortions.While the PSDM captures the degree of distortion in a particular photoreceptor signal, it does not directly describe the directionality of this distortion in relation to other photoreceptor signals, as the PCDM does.For example, it is possible for the PSDM to be non-zero while the PCDM is zero, if, for instance, all of the reproduced melanopsin signals are consistently less than the real-world melanopsin signals.The PCDM quantifies the distortion between the real-world correlation coefficient, 1 , 2 , between two photoreceptor signals and the reproduced correlation coefficient, * 1 , 2 , between those photoreceptor signals for all realisable spectra (Eq.8).

Equal-luminance photoreceptor excitation diagram
While the CIE 1931 xy chromaticity diagram provides a familiar and quick method to understand the chromaticities of a dataset it provides no information of the distribution of rod and melanopsin signals.It is further worth noting that the CIE 1931 xy chromaticity space is no longer considered to be physiologically relevant, as it uses the CIE 1931 XYZ colour matching functions as its basis, which are not a linear transformation of the widely used, and CIE 2006 standard, cone fundamentals [31][32][33]40].
For a physiologically-relevant chromaticity space, one can turn to the MacLeod-Boynton chromaticity diagram [1].While less common in the display industry, the MacLeod-Boynton chromaticity diagram is a widely-used colour space among vision scientists.The MacLeod-Boynton space provides a visualization of the cone excitations at equal luminance.The traditional MacLeod-Boynton chromaticity space plots MB ( ) against MB ( ), where MB ( ) and MB ( ) are defined as in Eq. 9: where ( ) = ( ) + ( ) and ( ) and MB ( ) peak at unity.(9) Here, the CIE 2006 10 • spectral sensitivity functions (as in the CIE S026 standard) are used and normalised to unity peak [31][32][33][34].In terms of energy, the scaling constants, , , and , take the following values [1,32] The spectrum locus in the traditional MacLeod-Boynton diagram is shown in Fig. 3(a).The MacLeod-Boynton space lends itself to a natural extension to include a melanopsin axis.Thus a new equal-luminance photoreceptor excitation diagram is proposed, which plots the melanopic efficacy of radiance i.e. the melanopsin excitation normalized by luminance [33].The newly proposed axis plots the variable MB ( ), which is defined in Eq. 10: scaled so that MB ( ) peaks at unity.(10) The CIE S206 spectral sensitivity function for melanopsin is used here, and normalised to unity peak [17,34].In terms of energy, the scaling constant, , takes the following value in order for MB ( ) to peak at unity: The spectrum locus is depicted in the proposed equal-luminance photoreceptor excitation diagram in Fig. 3(b)-(c).It is worth noting that the newly proposed equal-luminance photoreceptor excitation does not provide any visualization of the rod excitations.Previous work has been done to derive visualizations of rod excitation on traditional chromaticity diagrams [3], so in this paper we opted for an approach that allowed for visualization of the melanopsin signal.As the rod and melanopsin signals are very highly correlated (Fig. 2), and there is no known dominant post-receptoral pathway that compares rod and melanopsin signals, choosing to depict one of the rod and melanopsin signals allows for a three dimensional visualization of the five dimensional space that captures most of the relevant information necessary to understand the scope of the space.

Results: Example applications of novel metrics
To demonstrate the effectiveness of the proposed metrics they are applied here to three different real three-primary displays and to two hypothetical five-primary displays.
Three real three-primary displays are evaluated: a CRT NEC Color Monitor MultiSync FP2141SB (NEC Display Solutions, Tokyo, Japan), a CRS Display++ LCD Monitor (Cambridge Research Systems, Rochester, UK), and a Dell U2715H LCD Monitor (Dell, Austin, TX, USA).The displays were measured with a CRS SpectroCal MKII Spectroradiometer (Cambridge Research Systems, Rochester, UK).The measured primaries of the displays are shown in Fig. 4(a)-(c).Two hypothetical five-primary displays are also evaluated: a hypothetical narrowband display, and a hypothetical broadband display (primaries depicted in Fig. 4(d)-(e)).

Photoreceptor Signal Reproduction Metric (PSRM)
The majority of chromaticities of the real-world signals fall within the gamut of the chosen displays, as depicted in the CIE 1931 xy chromaticity diagram in Fig. 5(a).In fact, as shown in Fig. 5(b), at least 90% of the chromaticities can be reproduced by each display (CRT NEC Color Monitor: 90.8%; Dell U2715H LCD Monitor: 91.0%; CRS Display++ LCD Monitor: 94.1%; Narrowband Five-Primary Display: 100%; Broadband Five-Primary Display: 100%).Despite the excellent chromaticity reproduction, Fig. 5(c) shows the extremely low PSRM of the three-primary displays (CRT NEC Color Monitor: 1.1%; Dell U2715H LCD Monitor: 1.4%; CRS Display++ LCD Monitor: 1.9%).This is not surprising as any match between the reproduced melanopsin and rod signals to the real world signals is by coincidence.For both the hypothetical five-primary displays on the other hand, the PSRM is quite high (Narrowband Five-Primary Display: 70.5%; Broadband Five-Primary Display: 79.8%).The PSRM does vary between the hypothetical five-primary displays which shows that while having five primaries is necessary to reproduce the five photoreceptor signals, it is not sufficient to use any arbitrary five The primaries of the hypothetical narrowband five-primary display (all primaries with a full-width half-maximum of 10 nm and peaks at 450 nm, 500 nm, 550 nm, 600 nm, and 640 nm).(e) The primaries of the hypothetical broadband five-primary display (all primaries with a full-width half-maximum of 40 nm and peaks at 450 nm, 500 nm, 550 nm, 600 nm, and 640 nm).
primaries and expect to reproduce all desired real-world signals; similar to how one cannot just pick any arbitrary three primaries and expect to reproduce a large colour gamut in three-primary displays.The placement of primaries along the wavelength axis is critical.

Photoreceptor Signal Distortion Metric (PSDM)
By definition, for all spectra that fall within gamut, the PSDM for all photoreceptors for the two hypothetical five-primary displays is 0. Fig. 6(a) demonstrates the rod and melanopsin distortions introduced in the Dell LCD (taken as an example representation of the three-primary displays) when the cone signals are matched for all realisable spectra, by plotting the reproduced signal, * , against the real-world signal, , for each photoreceptor.When the reproduced signals match the real-world signal then all of the data points should fall along the unity line, as they do for the cone responses in the three-primary displays and for all of the photoreceptor signals in the five-primary displays (not shown).Fig. 6(b) shows the distribution of the melanopsin and rod PSDM across the realisable spectra (i.e. the spectra which fall within gamut) for the Dell U2715H LCD Monitor.Fig. 7(a) and Fig. 7(c) show the box-plot distributions of the melanopsin and rod PSDM of the CRT NEC Monitor and CRS Display++ LCD Monitor.For all three three-primary displays the PSDM for both melanopsin and rods is widely spread (CRT NEC Color Monitor: PSDM = 16.0%,PSDM = 24.8%;Dell U2715H LCD Monitor: PSDM = 15.3%,Despite the high chromaticity reproduction of all displays, the PSRM of the threeprimary displays is very low, while it remains quite high for the five-primary displays.PSDM = 23.6%;CRS Display++ LCD Monitor: PSDM = 13.6%,PSDM = 20.2%).For the CRT NEC Color Monitor and Dell U2715H the mean PSDM for both melanopsin and rods is large (CRT NEC Color Monitor: PSDM = 14.1%,PSDM = 20.2%;Dell U2715H LCD Monitor: PSDM = 10.4%,PSDM = 15.2%).For the CRS Display++ LCD Monitor, the mean PSDM for both melanopsin and rods is near zero (CRS Display++ LCD Monitor: PSDM = 0.9%, PSDM = 1.1%) though it is worth noting that the spectra that can be reproduced with near zero rod distortion introduce melanopsin distortion, as indicated by the low PSRM for all three three-primary displays.Further, when considering the mean of the absolute value of the PSDM for each photoreceptor for each three-primary display, one can see the magnitude of the mean distortion introduced is non-zero (CRT NEC Color Monitor: |PSDM| = 16.3%, |PSDM| = 23.5%;Dell U2715H LCD Monitor: |PSDM | = 13.8%,|PSDM| = 20.1%;CRS Display++ LCD Monitor: |PSDM| = 10.1%, |PSDM| = 14.7%).For the CRT NEC Color Monitor the overall PSDM is 7.97%, for the Dell U2715H LCD Monitor the overall PSDM is 6.79%, and for the CRS Display++ LCD Monitor the overall PSDM is 4.96%.
The melanopsin and rod PSDM shows a chromaticity dependence.This is not surprising as the photoreceptor signals are highly correlated.To illustrate the chromaticity dependence of the PSDM the CIE 1931 xy chromaticity diagram is divided into a 100 × 100 grid (with each bin covering 0.01 in x and 0.01 in y), and the mean of all of the melanopsin and rod PSDM values for any real-world spectra whose chromaticities fall within one grid square on the divided diagram is taken as the average PSDM and used to colour that grid square.This is illustrated for the Dell U2715H LCD Monitor in Fig. 6(c), and shown for the CRT NEC Monitor and CRS Display++ in Fig. 7(b) and (d).A similar principle is applied to the MacLeod-Boynton chromaticity diagram, whereby the chromaticity space is divided into a 100 × 100 grid and the mean melanopsin and rod PSDMs.An example is illustrated for the Dell U2715H LCD Monitor in Fig. 6(d).

Photoreceptor Correlation Distortion Metric (PCDM)
By definition, the hypothetical five-primary displays have a PCDM of zero for all photoreceptor pairs.Fig. 8 shows the PCDM for all photoreceptor pairs introduced by the three three-primary displays across the realisable spectra for each display.It is worth noting the large correlation distortions, particularly between the three cone excitations and the rod and melanopsin excitations, introduced by all of the three-primary displays.The fact that the PCDM is non-zero indicates that it is not simply the case that all of the reproduced melanopsin excitations are less than the real-world melanopsin excitations, or in other words that the distorted photoreceptor signals are being mapped onto one of the undistorted photoreceptor signals.The PCDM for different photoreceptor pairs has different signs, showing that the correlations between different pairs of photoreceptors are sometimes increased and sometimes decreased.This means that not only will some of the correlations between real-world signals be lost during reproduction, but spurious correlations will be introduced by reproduction that do not exist between the photoreceptor excitations in the real-world.

Equal-luminance photoreceptor excitation visualizations
The 39,699 real-world spectra are plotted in the proposed equal-luminance photoreceptor excitation diagram in Fig. 9(a).As in the depiction of the PSDM, the space is divided into a 100 × 100 grid, and the frequency of the real-world spectra falling within a square on the grid is shown as the intensity of the heat-map.Any spectra that are considered reproduced on a particular display (as in the PSRM), are depicted on the proposed equal-luminance photoreceptor excitation diagram space in Fig. 9(b).In the case of the five-primary displays, the reproduced spectra look very similar to the real-world spectra in the proposed equal-luminance photoreceptor excitation diagram space, though covering a slightly reduced area due to the non-perfect reproduction (i.e. the PSRM is not 100%).For the three-primary displays however, the reproduced spectra on the proposed equal-luminance photoreceptor excitation diagrams cover a patchy and reduced area of the chromaticity diagram, indicating the inability of three-primary displays to reproduce all five real-world photoreceptor signals.

Discussion
Until now, display reproduction and metrics to characterize display reproduction have been focused solely on describing the reproduction of the visual signals of the image being reproduced.Colour gamuts have long been recognised as an important quantification of display reproduction, but they only describe chromaticity reproduction i.e. the reproduction of the cone responses.With mounting evidence suggesting that melanopsin strongly contributes to driving the non-visual responses to light, and that rod saturation may not be as straightforward as once assumed, the display industry must begin to consider the reproduction of rod and melanopsin responses on displays.To do so, there need to be metrics to describe a display's ability to reproduce, or in the case of displays that do not directly control the signals, extent to which they distort, all five photoreceptor signals.Three such metrics were proposed here.

The applicability of the proposed metrics
The three novel metrics should provide information about an image reproduction system's ability to capture all classes of photoreceptor signals available in the retina.The photoreceptor signal reproduction metric (PSRM) describes how many of the five photoreceptor signal quintuplets from a reference dataset can be reproduced on a display in a non-distorted manner, i.e. in the same ratios as in the real-world spectra.The photoreceptor signal distortion metric (PSDM) describes what distortions are introduced when the relative photoreceptor quintuplet is distorted during the reproduction on a display using fewer than five primaries.Finally, the photoreceptor correlation distortion metric (PCDM) quantifies how the correlations between the photoreceptor signals from the reference database are distorted by the display.Additionally, an equal-luminance photoreceptor excitation diagram is proposed as a visualization of signals that drive the human physiological response to light is proposed, which should provide a useful way to illustrate where distortions are introduced and where display reproduction fails.
As interest in "night-shift" modes, 3D displays, and real-world reproduction continues to grow in the display industry, the consideration of melanopsin signal reproduction by display manufacturers becomes increasingly important.It is hoped that the metrics proposed here highlight the fundamental limitation of three-primary systems to control the non-visual physiological response to light, due to the lack of control and resulting distortions of the melanopsin signal.It is further hoped that the metrics and visualization proposed here offer a useful method to characterize an image reproduction system's ability to accurately reproduce the full set of photoreceptor signals from a real-world scene.It is worth noting that the metrics proposed here are not being suggested as a replacement to current metrics, such as colour gamuts, but rather as an addition to current metrics.

Limitations of proposed metrics
There are limits to what the three metrics proposed here can quantify.One notable limitation is that the metrics are based on low-level physiological information, rather than on higher-level perceptual, circadian, and cognitive information.The metrics characterize the physiological signals available to the retina, rather than the physiological, psychology, and behavioural responses driven by these signals.One could imagine an analogue of a colour appearance model for circadian rhythm shift, or mood control, for example.However, much more work is needed to quantify how the photoreceptor signals are combined to drive the non-visual features of light for such metrics to be developed.More work would be needed to quantify how a display that severely distorts melanopsin signals compares against a display that only minimally distorts melanopsin, and one that reproduces melanopsin signals exactly, in terms of, for example, the shift to circadian cycle introduced in such cases.For now the metrics proposed here, which directly describe the physiologically relevant signals available to the retina are an appropriate quantification of displays while the physiologically relevant mechanisms driven by such signals remain elusive.
A further limitation of the current metrics is the lack of consideration for absolute intensity.The proposed metrics do not take into consideration whether the relative photoreceptor signal quintuplet can be presented at the same intensity as they are found in the real-world spectra.Absolute intensity is not considered here as there is merit to quantifying the distortions introduced in the rod and melanopsin signals when the relative cone excitations can be reproduced, even when absolute intensity cannot be e.g. in the case of reproducing the daylight spectrum on a standard LCD monitor.If one wanted to incorporate absolute intensity into the metrics, then one could define the PSRM such that a photoreceptor quintuplet is only considered to be reproduced if it is reproduced at the same intensity as the real-world quintuplet.Alternatively one could report the proposed metrics alongside the intensity range of the display or quantify the absolute intensity range over which the three metrics proposed here apply.
A challenge of incorporating absolute intensity into the metrics proposed is knowing the absolute intensity range over which different real-world spectrum are likely to exist i.e. a better characterization of the "spectral diet" most people experience [44].The real-world reference dataset used here was chosen as the reference because it represents surfaces and illuminants that one may typically encounter in the real-world, and thus radiant spectra that one may wish to reproduce on a display.However, the current real-world spectra dataset does not weight the samples by likelihood of encountering a particular sample.More data are needed to understand the frequency of spectral encounters in the real-world [44,48].The metrics described in this paper do not rely on the choice of any particular reference dataset, just on a suitable reference dataset for the reproduction purpose.As more is understood about the spectral diet of humans, the reference dataset for reproduction should be updated.It is also worth noting that a universal real-world reference set as used here may not be appropriate for all reproduction activity.For instance, if one wished to artificially decrease melanopsin signal in "night-modes" of displays then one may wish to define an altered real-world reference set to suit one's purpose.
A further challenge for display reproduction and quantification of display reproduction is the range of individual differences that exist in the photoreceptor spectral sensitivity functions, even amongst the normal vision population.Differences in cone spectral sensitivity functions have long been of concern for display manufacturers, as extreme differences can result in metameric failure, i.e. what is a colour match on a display to a real-world stimulus for one individual is not a colour match for another.Just as with the cone spectral sensitivities, one would expect differences in the rod and melanopsin spectral sensitivity functions driven by differences in lens, macular pigment, and optical density, though a characterization of the variance of melanopsin and rod functions remains to be derived, and at present only the standard functions are used.This means that even in displays where melanopsin and rod signals are reproduced without distortion for the standard observer, they may be distorted for different individual observers, though it is unclear how large these distortions would be in practice, and the impact that small distortions would have on the reproduction of non-visual features of light.

The choice of primaries for five-primary displays
While five-primary displays offer an obvious means to control the melanopsin signal directly, it is worth noting that the choice of primaries is important.Even in the hypothetical five-primary displays proposed here, the PSRM is below 80%, indicating that while having five primaries is necessary to accurately reproduce the full photoreceptor quintuplet, five arbitrary primaries are not sufficient to reproduce any arbitrary photoreceptor quintuplet (or more so, some of the photoreceptor quintuplets encountered in the real-world).To address this it would be necessary to carefully optimise selection of primaries to best reproduce the photoreceptor quintuplets of interest.

Conclusion
In this paper three novel, physiologically relevant metrics to characterize the full photoreceptor response to light are proposed, along with a visualization beyond chromaticitiy space of the photoreceptor response in the form of a new equal-luminance photoreceptor excitation diagram (an extension of the MacLeod-Boynton chromaticity diagram).These methods allow for the quantification of photoreceptor signals beyond the cones to describe display reproduction, and also illustrate the fundamental problem of photoreceptor signal distortions introduced by three-primary displays.The metrics and visualization proposed here are not envisioned to be the only metrics used in the future to quantify the full human response to light from displays, but more a starting point to begin to describe the often forgotten about photoreceptor signals in display metrics.

Fig. 1 .
Fig. 1.An example of how a display may reproduce and distort photoreceptor signals.(a)The spectral sensitivity functions of the cones, rods, and melanopsin (as in the CIE S026 standard)[17,31,32,34,35]. (b) An example radiance spectrum from a real-world reflectance surface viewed under a real-world illuminant.(c) Example primaries of a hypothetical three-primary display.(d) The spectral output of the hypothetical three-primary display (c) that reproduces the same cone excitations as the real-world radiance spectrum (b).(e) The cone, rod, and melanopsin excitations from the real-world radiance spectrum ((b), with black outline) and from the hypothetical three-primary display's spectral output ((d), grey outline).(f) Example primaries of a hypothetical five-primary display.(g) The spectral output of the hypothetical five-primary display (f) that reproduces the same photoreceptor excitations as the real-world radiance spectrum (b).(h) The photoreceptor excitations from the real-world spectrum (b, black outline) and the hypothetical five-primary display's spectral output (g, grey outline).One can see that while the rod and melanopsin excitations from the real-world spectrum and hypothetical three-primary display output do not match even though the cone excitations do, all five photoreceptor signals match between the real-world spectrum and hypothetical five-primary display.

1. 4 .
Quantifying a display's physiologically relevant reproduction of light An immediate problem arises when one begins to seek solutions for controlling melanopsin and rod responses from image reproduction systems: what metrics should one use to quantify the distortion and reproduction of the melanopsin and rod signals?Colour gamuts based on the CIE 1931 XYZ functions, by definition, only capture the chromaticity, i.e. the cone signals, that a display can reproduce [40].The CIE 2006 (170-2) introduces a move away from the CIE 1931 XYZ functions towards a physiologically relevant colour space, based on the CIE 2006 (170-1)

Fig. 2 .
Fig. 2. The correlations between the photoreceptor excitations from the 39,699 realworld spectra.(a) The photoreceptor excitations from the real-world spectra plotted pairwise.(b) The Pearson correlation coefficient matrix plotted as a heat-map showing the correlation coefficient of each pairwise combination of photoreceptor signals.

Fig. 3 .
Fig. 3.The proposed equal-luminance photoreceptor excitation space to capture melanopsin signals.(a) The spectrum locus plotted in the traditional MacLeod-Boynton chromaticity diagram, from 390 nm to 780 nm.Crosses indicate 25 nm steps and are coloured using a perceptually uniform rainbow colour-map to indicate the progression from short to long wavelengths along the spectrum locus [47].The daylight locus, from 4, 000K to 14, 000K is also shown, with a blue to yellow colour-map depicting the progression from cooler to warmer temperatures along the daylight locus.(b) The spectrum locus and daylight locus plotted in one projection of the proposed equal-luminance photoreceptor excitation diagram, plotting the melanopic efficacy of radiant flux (I/(L+M)) against the L/(L+M) excitation.(c) The other projection of the proposed equal-luminance photoreceptor excitation diagram, plotting the spectrum locus as S/(L+M) against I/L+M.

Fig. 4 .
Fig. 4. The display primaries used as an example application of the proposed metrics.(a) The primaries of the CRT NEC Color Monitor.(b) The primaries of the Dell U2715H LCD Monitor.(c) The primaries of the CRS Display++ LCD Monitor.(d)The primaries of the hypothetical narrowband five-primary display (all primaries with a full-width half-maximum of 10 nm and peaks at 450 nm, 500 nm, 550 nm, 600 nm, and 640 nm).(e) The primaries of the hypothetical broadband five-primary display (all primaries with a full-width half-maximum of 40 nm and peaks at 450 nm, 500 nm, 550 nm, 600 nm, and 640 nm).

Fig. 5 .
Fig. 5.The chromaticity reproduction and PSRM of the real-world spectra on the example displays.(a) The 39,699 spectra plotted on the CIE 1931 xy chromaticity diagram, along with the colour gamuts of the example displays.(b) The chromaticity reproduction rate of the example displays.(c) The PSRM of the example displays.Despite the high chromaticity reproduction of all displays, the PSRM of the threeprimary displays is very low, while it remains quite high for the five-primary displays.

Fig. 6 .
Fig. 6.The PSDM applied to the Dell U2715H LCD Monitor as an illustrative example.(a) The rod and melanopsin photoreceptor signals reproduced by the display are plotted against the rod and melanopsin signals from the real-world spectrum.In the case the signals are undistorted, then they should fall along the line of unity (as is the case for the cone signals, which are not shown).(b) Box-plots showing the distribution of the rod and melanopsin signal distortion (as defined in Eq. 7) across the three displays.(c) The mean rod and melanopsin distortion across the CIE 1931 xy chromaticity diagram.The distortions from all of the photoreceptor signals whose chromaticities fall within a square on the evenly spaced 100 × 100 grid (with steps of 0.01 in both x and y) of the chromaticity diagram are taken together and the mean displayed in this plot.The black dot indicates D65.(d) The mean rod and melanopsin distortion across the MacLeod-Boynton chromaticity diagram.Again, all photoreceptor quintuplets who fall within a particular bin (step size of 0.01 in both the L/(L+M) and S/(L+M) directions) in the 100 × 100 grid across the diagram are taken together and the mean rod and melanopsin distortion found (black dot shows the chromaticity of D65).

Fig. 7 .
Fig. 7.The PSDM of the CRT NEC Color Monitor and the CRS Display++ Monitor.(a) Box-plot showing the distribution of the rod PSDM.(b) The rod PSDM across the CIE 1931 xy chromaticity diagram.(c) Box-plot showing the distribution of the melanopsin PSDM.(d) The melanopsin PSDM across the CIE 1931 xy chromaticity diagram.The black dot on the heat-maps shows the chromaticity of D65.

Fig. 8 .
Fig. 8.The PCDM of the three three-primary displays: the CRT NEC Color Monitor, the Dell U2715H LCD Monitor, and the CRS Display++ Monitor.By definition, the PCDM between the cone signals is 0, but large distortions are introduced in the correlation between the rod and melanopsin signals with each other photoreceptor.

Fig. 9 .
Fig. 9.The real-world spectra and reproduced display spectra depicted in the proposed visualization method of the extended MacLeod-Boynton space, along the traditional axis, and the two new projections including the extended axis.(a) The real-world spectra plotted as a heat-map, depicting the number of real-world spectra that fall within a particular square when the axes shown are divided into a 100 × 100 grid.(b) The spectra that can be reproduced on the various monitors, depicted the same way (with the same axes and colorbar) as the real-world spectra in (a).Only the spectrum that can be reproduced without any distortion in any of the photoreceptor signals i.e. = * for all ∈ [ , , , , ], are included in the plots in (b).