Interpretation of performance metrics

[Cui Hong]

Dear all,

I have a question on how to interprete eMotivPro performance metrics.

Say, I have N stimuli, which are shown to a study participant one by one. For each stimulus, the participant reports their perceived stress level, and performance matrics report EEG detected stress level. Is it meaningful to do a Spearman correlation test between the two scores for the N stimuli? Will this correlation tell us to what extent EEG and human perception of stress match?

Thank you for sharing your thoughts/comments/suggestions!

 

 

• This topic was modified 2 years, 9 months ago by Cui Hong.

[luckham jerill]

Hello Cui,

Yes, this kind of experiment can yield information about perceived stress and underlying stress. Simple correlation may be adequate, depending on confounding factors, intensity and duration of the stimuli etc. Other anaysis types (Anova, Manova etc) may be required. The researcher may need to account for some latency in the onset of stress. Our detections have some inherent latency (0.5-~3 seconds), and there may also be a delay in physiological onset of stress. May I know which headset you are using please?

[Cui Hong]

Thank you Luckham for your response. We were using Insight.

[Cui Hong]

Another puzzle I try to figure out is about the performance metrics during the close-eye and open-eye calming-down periods. In the image linked here (https://www.screencast.com/t/4MFgl8m0IJ), the calming-down periods are the first 36 seconds (white background). The grey line=interests, the orange line=stress, and the blue line=engage. I expected all three lines to be low during the calming down periods, but EmotivPro shows a very high interest (actually higher than the interest level invoked by all stimli tested, the referenced image does not show the complete interest line). Can you help me understand this? Thank you!

[Cui Hong]

Related, even though I have read the description for the scaled and raw metric data in https://emotiv.gitbook.io/emotivpro/exported_data_files/csv_files multiple times, I am not sure I really get it.

I expected the correlation between raw and scaled data (exported from EmotivPro) to be really high (like above 0.9) because they are alternative representations of the same event, but I got only 0.7 from several sets of data (see for example the data and graph in https://www.screencast.com/t/2MbZkserYM). In the graph, I found a number of places (marked with arrows) where the direction of change is different between raw and scaled data (one goes up while the other goes down as compared to the neighboring points).

Can anyone help me understand under what kind of circumstance would these kinds of differences happen? When you perform correlation analyses, should you use raw or scaled metrics?

Your help is much appreciated.

 

[luckham jerill]

Hello!

Thank you for your questions. Please clarify if you are using high-resolution Performance Metrics or the low-resolution. May we also have your x-axis points?

On the second chart (blue & orange line PM) what type of data is “raw” & “scaled”?

Looking forward to your follow up

[Cui Hong]

HI, I believe we used low-resolution PM because we got 1 PM score every 10 seconds in the exported csv file.

In https://www.screencast.com/t/2MbZkserYM, “raw” and “scaled” refer to the raw and scaled PM scores found in the exported csv file (as described in https://emotiv.gitbook.io/emotivpro/exported_data_files/csv_files).

 

[luckham jerill]

Hello Cui,

Thank you for your reply.
Raw performance metrics are usually values that are difficult to interpret as they are not 0-1 values. The reason the scaled values and the raw values are not exactly correlated is because the auto-scaler adjusts & changes in accordance to the min and max values as the recording goes on. This enables the LIVE output to be a better representation of the individuals personal “scale” of stress/focus/engagement relative to the rest of the recording. However, for scientific research and accurate correlation analysis, it is recommended to retrospectively rescale the PM’s when the recording is finished (on export). This can be done using the final max and min values of the PM. Find the last possible values of MAX and MIN where CQ has been continuously usable. Call them MAX_F and MIN_F. MIDDLE_F = (MAX_F + MIN_F) / 2
RANGE_F = ABS( MAX_F – MIN_F )
For each PM_Raw, calculate:
PM_Retro = 1 / ( 1 + exp ( -5 * ( PM_Raw – MIDDLE_F ) / RANGE_F ). @jerill
You can also scale to a baseline (eyes open) condition or express the PM values as a % of baseline levels. (condition – baseline) / baseline to understand how the performance metrics move from baseline levels.

[Cui Hong]

Thank you for the information.

Just to confirm my understanding… So the values in the ‘scaled’ columns exported from EmotivPro are NOT the ‘final’ scaled values — they were scaled during the recording and were using the MIN and MAX values up to that moment.  After a study session completes, we need to use the data in the ‘raw’ columns to re-compute the scaled values with the ‘global’ MIN and MAX. Right?

Honestly, I do not see why EmotivPro does not do this ‘final’ scaling at the time of data export and includes the final scaled values in the exported CSV file. What is the use of the moment-by-moment ‘scaled’ value in the exported CSV file?

So if I conduct this final scaling on my raw values, what would the correlation between my raw and final scaled values be, according to your experience? I am new to EEG and would appreciate your expert’s opinion as to what to expect. Also do you recommend using scaled values to assess the correlation between a subject’s EEG values and their self-reported stress levels? Since it is within-subject, I feel using raw or scaled should not make a big difference (actually, prefer raw more than scaled).

Again, thank you for answering my questions.

[luckham jerill]

Hello,

Thank you for your reply.

Yes, you are correct. You still need to use the data in the ‘raw’ columns to re-compute the scaled values with the ‘global’ MIN and MAX.

Regarding EmotivPRO prividing the “final” scaling, this is a good suggestion. We considered it, but it may be confusing for people who recorded specific values of each PM at different times during the live recording. We decided it was better to export the data as originally displayed, but provide a means to rescale it to a more consistent scale.

The correlation between rescaled and raw values is very high. The rescaling formula is logistic, which maps -inf to +inf onto the 0-1 range. Outlying data points are compressed into the top and bottom few percent of the 0-1 scale, while retaining a strict monotonic mapping. That is, if raw(t1) > raw(t2) then rescaled(t1) > rescaled(t2) for all t1, t2

Whether you choose raw or rescaled data depends on the requirements of your analysis. Typically different subjects have widely differing range and and baseline values for the same performance metrics. The differences are due to both physical and psychological differences.

Anatomically, every brain is folded differently. Specific functional areas are not perfectly mapped onto the same regions of each cortex. Many major functions are located in ‘similar’ regions for anatomical or developmental reasons, but no two brains are mapped identically due to fairly chaotic developmental processes. The huge cortical area is densely folded to fit inside the very finite skull volume. Different fold patterns and deep, interlocked folds mean that any specific functional region of the cortical surface may be deeply buried in tissue or presented on the outer surface of the cortex, Differing skull thicknesses, location of muscles, blood vessels etc also contribute to wide variations in scale of different PMs.

Different personalities also express different ranges of emotions and different “operating levels” – think of the difference between an ex-fighter pilot who can calmly land a dead passenger plane on the Hudson River, and a cab driver (we’ve all had one) whose natural state is yelling at the traffic and gets enraged by someone daring to ride a bicycle on a city street.

If the target of your research is to compare between different types of stimulus across a population, it is more appropriate to use standardised data such as the rescaled PMs. If you are more interested in differences between individuals, then (perhaps) raw values are more appropriate.

Hope this helps. If you have other questions, please let me know.

[Author]

Posts