Follow-up Study: Discussion

[DominiqueMakowski]

@MarcoViola88 @guidocor @AleAnsani @AntonioOLR84 @marmarini @marcosperduti

Here's the thread to start discussing a potential follow-up while we dissect the current study.

1. Preregistration

• Don't forget

2. Mobile users

• Make the task more mobile-friendly? Complicated + not needed for prolific
• Do a specific mobile population (assuming the task is made more mobile-friendly) to replicate findings (bigger effect?)
  • Depends on budget (alternatively, can be moved to next year to recruit students)
• Add "where you alone check"

3. Change Stimuli

• Use new database?
  • Low quality + duplicates
  • Problem with making the study fully open
• "Too many irrelevant"?
  • Filter stimuli based on sex & orientation?
• Not enough arousing
  • Add couples?
    • Only hetero couples stims?

4. Improve manipulation (warranted by the high rate of "i think the labels were incorrect")

• Improve and emphasize the cover story?
• Implicit manipulation
• Different dimensions of reality
  • "we should clarify what "dimensions" of reality we aim to manipulate and investigate. For instance, 2 dimensions that we can manipulate (in an image-based paradigm) could be something akin to "authenticity" (actors/distant/professionals vs. real/relatable/next-door-neighbour-people) and "artificiality" (the simulated nature of the content, i.e., real photos vs. AI-generated). They might involve slightly different psychological underpinnings, for instance, the former could be related to the notion of "psychological distance" & self-relevance, and the second to simulation monitoring."
  • change scales:
    • how much time would you like to spend watching?

5. Reality scale: phrasing open to interpretation

• Change the second phase to half-reveal the manipulation ("actually, labels were at random, try now to assign it yourself") and explicitly ask participant to tell wether they believe the images are real/ai-generated?

6. Moderators

• Assess the usefulness of current covariates to confirm those to re-use, those to drop and those to potentially add (e.g., personality)
• Deeper into Sex/Porn/horniness as female moderator
• Improve outlier control
  • Include explicit attentional checks
  • Filter based on instruction page reading time

7. Role of beliefs / Metacognition

• Explicitly manipulate beliefs via different groups where we suggest that "Research has found that feeling less/more aroused by AI-generated was associated with a better mental health"
• Task demand characteristics: Task demand characteristics: ask participants:
  • "I think I found ai generated pictures less arousing" vs. "I think in general people find AI pics less arousing" (did they know the hypothesis?)

[guidocor]

Antonio made me a summary of the meeting.

Preregistration
Don't forget

I think that we could just use the current data as pilot for preregister and preregister the Prolific participants. I usually made this kind of things for preregistering experiments, specially when linear miced models are involded (we made a pilot to estimate correlation parameters for the simr simulations)

"Too many irrelevant"?
Filter stimuli based on sex & orientation?`

Filter could induce problems as modelling becomes more complicated. But, choosing only hetero people could be disciminatory!

[MarcoViola88]

Hi guys, not a real "metholodogical brainstorming", but still a good news: I checked with the administration and apparently I might be able to get some fundings for Prolific. Cannot invest more than 400€ as of now, but I'll try to find some further sources...

[DominiqueMakowski]

✋
401€ for the gentleman over there
Any other bid for this wonderful study in mint condition with a ravishing and intricate baroque methodology?

---

In case that's helpful:

https://www.prolific.com/calculator

It's roughly 10 participants / 50€

[guidocor]

Hahah I have 257 GBP in my Prolific account and I think I can have somo 200 more

[MarcoViola88]

I just realized that OASIS, an Open Access validated database for images including Valence & Arousal scores (Kurdi et al. 2017), includes several pictures of nude men, women, and (heterosexual) couples.
We may consider picking some images from it in case we want to differentiate Study 2 a little bit

https://www.benedekkurdi.com/%23oasis
Kurdi, B., Lozano, S., & Banaji, M. R. (2017). Introducing the open affective standardized image set (OASIS). Behavior research methods, 49, 457-470.

[DominiqueMakowski]

If I remember the images are quite low quality in terms of format & size standardization but I might be misremembering

[MarcoViola88]

I am afraid you remember correctly: the quality is lower here than in EroNAPS. Still, maybe they might be a "plan B" if necessary?
In any case, since they have different qualities the only thing we should probably AVOID is to MIX images from both databases.

On another note, I'm happy to see that we reached N=354. I should be able to collect some more participants from Italy, but maybe it's almost time to restart brainstorming about the Exp 2? :-D

[DominiqueMakowski]

maybe it's almost time to restart brainstorming about the Exp 2? :-D

Yes, absolutely, on my side there is a new student that will be helping with this and they might have some timelines so I'll quite likely start some thread soon (probably in ~2 weeks, after the break) 😏

[marcosperduti]

"Italia prima in classifica per video porno con trans. “Cercati nel 100,5% dei casi in più rispetto alla media mondiale" (La Repubblica)
"Italy first in the ranking for trans porn videos. "Searched in 100.5% more cases than the world average" (La Repubblica)

We probably should try this for the Italian sample in the next study 😏

[MarcoViola88]

Hi everybody, remember that @guidocor flagged an interesting new database? (described here: https://link.springer.com/article/10.3758/s13428-024-02418-z)

Well, I reached the authors and asked access to the database -- if that's nice, perhaps we might use it for our 2nd round.

I hereby quote their reply + my comments.

To download the EPPS, please send us an email to mpastor@uji.es with the following information:

A) Name of the research institution (e.g., university or college). Please also include the (1) name (2) mailing address and (3) website of the specific research department where the research will be conducted.

If you don't mind, I'd say something like: "We're doing it mainly online, but the homebase is Uni of Sussex" & providing all the relevant info of @DominiqueMakowski's department.

B) Name/s and email/s of the Principal Investigator/s (NOTE: Principal Investigator is the person who takes ultimate responsibility for the proper usage of the images. In other words, this person should not be a trainee or research assistant).

Based on the amount of work done, I'd say that the P.I. here is Dominique. But if that helps, I can indicate myself as a co-P.I.

C) A brief description of the research study and how the images will be used. Please include this information in the email body and the attached document.

Shouldn't be difficult, although I'd keep it pretty vague since we haven't had the brainstorming meeting - agree?

[Antonio8424]

Some brainstorming on AI & Moral Judgments (nudes/sexting/onlyfans)...

OnlyFans and webcam models often fear being “discovered” and judged by their closest relatives, such as family or work colleagues. This fear arises because publicly sharing personal sexual material may be considered inappropriate or even taboo in certain cultural contexts, potentially leading to serious consequences, like being fired. Additionally, individuals whose morality heavily relies on “purity” concerns—related to physical and spiritual contagion, chastity, control of desires, and linked to the emotion of disgust—may be particularly prone to morally condemning these practices (e.g., conservatives).
What about considering tracking the moral perception angle of the deepfake situation? It is somewhat obvious, but I would expect that suspecting someone is using AI in their OnlyFans content would reduce moral condemnation of the act (“it's not really them”). Paradoxically, althought this suspicion could morally absolve them, it could also diminish their success…as the content might be perceived as less arousing.

Some (even more) random thoughts:
At some point in the research, it might be interesting to apply an implicit approach to the impact of realness versus fakeness constructs in moral judgments of sexual stimuli (priming?).
I was wondering if identifying that people in the images use clearly identifiable facial filters could reduce sexual arousal, even when knowing that the person is real (e.g., onlyfans). Maybe not the easiest experimental manipulation, but just an idea.

[MarcoViola88]

Hi guys,
here are my (sketchy) notes about today's meeting.
Happy to provide clarification if asked!
Please don't miss the HOMEWORK section for everybody.

---

• WHO'S ONLINE? Ana, Antonio (left a little sooner), Dominique, Guido, Marco M, Marco S, Marco V

• recap of previous results --
  Small effect: for all values (Arousal/Enticement/Valence), photos > AI-generated
  … but: quite noisy, possibly moderated by other stuff (e.g., beliefs check)
  … also: BAIT scale seems useful (and distinct) & distinct from GAIIS

… see
--https://realitybending.github.io/FictionEro/analysis/2_emotions.html#summary
--https://realitybending.github.io/FictionEro/analysis/3_realness.html

---

EXP 2 -- Discussion following up on DM's previous post (#12 (comment))

1. PRE-REGISTRATION?
   -- don't forget pre-registration! Should be easier by now since we have a basic effect & knowledge of statistical models

2. MOBILE USERS?
   -- NB in Exp1 the effect is stronger for mobile users
   --- (WHY?) maybe because by noting less details they rely more on contextual knowledge
   --- (WHY?) … or maybe mobile user have better access to intimate xontext, whereas computer users
   -- Domnique has the expertise to make the exp more mobile-friendly
   -- on Prolific we can target non-mobile users

3. STIMULI
   -- many (hetero) males complained about seeing irrelevant
   -- maybe we can include (heterosexual) COUPLES? (rather than the irrelevant)
   --- good because they are more arousing & good because they are never irrelevant (for heterosexual)
   -- SAME database or Others?
   --- changing the DB is risky because what if we don't get results?
   --- we want to avoid the KINK
   --- we could probably mix them (20 relevant from NAPS + new EPPS + couples from both + some neutral)
   -- check presence / absence of a FACE, that might trigger Theory of Mind
   --- good news: we can take Eye-Tracking data with cameras (requires a further ethical approval)
   … Dominique is going to check validation data & make hypotheses about new stuff

[in the future, sounds too might o]

4. IMPROVE MANIPULATION
   -- many people had doubts (feedback: "I thought the labels were incorrect")
   --- (WHAT TO DO?) better work on the cover story? e.g., putting logos and bullshits about being engineering team collaborating with photographers ...
   --- (WHAT TO DO?) using watermarks or other elements which manipulate implicitly [FOR A FOLLOW-UP]
   -- PRIORITY: having OBJECTIVE measures
   --- … not necessarily "physiological": we can have a (2nd?) task like showing 2 blurred pictures and asking "which picture do you want to unblur"?
   --- with Prolific, we can track the originalparticipants a months later or so and get them doing a re-test
   -- ask people "were you alone/in intimate environment?"

5. REALITY SCALE
   -- we can confess the random manipulation ("we actually placed the labels at random") and ask them to judge them now

6. MODERATORS
   -- for women, porn usage seems correlated with the effect
   -- attention check in the questionnaire
   -- we should try to get some cues about which mechanisms are at play for the sake of performing follow-up studies (e.g., in an open-ended debriefing)

7. BELIEFS
   -- should we let people CREATE priors? (e.g., priming people with priors like "apparently people enjoy AI/photos more than photos/AI"?) [follow-up]
   --

PROLIFIC: we might need about 1500€ (1200£) for 400 participants, 20mins, 2.40£ per hour
Viola 600 €
Corradi 400 € (+ 300€)
Makowski 500 € ???

JOURNALS
Nature Human Behavior? Computers in Human Behavior? Scientific Reports? Collabra?

HOMEWORK:
-- think about the moderators !
-- think about the stimuli (good to mix databases? keeping neutral mages?)
-- think about the implicit/explicit manipulation
-- (@dominique) compare 2 databases
-- (@viola) continuing working on INTRO & Discussion
-- check how merging money on Prolific
-- (@marini) organize the party in Rome!

[marcosperduti]

[in the future, sounds too might o]

Why not short video? This would be far more arousing. Obviously for the moment there is an issue of credibility (can AI generate convincing erotic videos?), and it needs a lot of work of selecting and validating the stimuli (even if there are papers using erotic video, see below). That being said, I think the use of video would be much more "ecological".

4. PRIORITY: having OBJECTIVE measures
--- … not necessarily "physiological": we can have a (2nd?) task like showing 2 blurred pictures and asking "which picture do you want to unblur"?
--- with Prolific, we can track the originalparticipants a months later or so and get them doing a re-test

To follow up on this idea, I found some relevant works:

• Quinsey, V. L., Ketsetzis, M., Earls, C., & Karamanoukian, A. (1996). Viewing time as a measure of sexual interest. Ethology and Sociobiology, 17(5), 341-354.
• Lalumiere, M. L., Babchishin, K. M., & Ebsworth, M. (2018). The use of film clips in a viewing time task of sexual interests. Archives of Sexual Behavior, 47(3), 627-635.
• Wilson, R. J., & Miner, M. H. (2016). Measurement of male sexual arousal and interest using penile plethysmography and viewing time. Treatment of sex offenders: Strengths and weaknesses in assessment and intervention, 107-131.
• Israel, E., & Strassberg, D. S. (2009). Viewing time as an objective measure of sexual interest in heterosexual men and women. Archives of Sexual Behavior, 38, 551-558.

It seems that there is a large body od literature showing a "viewing time effect" that is preferred sexual stimuli are viewed longer in free viewing condition, and in some studies the viewing time correlate with subjective reports.

One issue raised by @DominiqueMakowski is that allowing viewing time vary across stimuli could be problematic. The study below shows that even in condition of restricted viewing time (all pictures are presented the same time) response time to rate sexual arousal is longer for preferred sexual stimuli, so response time could be a proxy of viewing time.

• Imhoff, R., Schmidt, A. F., Nordsiek, U., Luzar, C., Young, A. W., & Banse, R. (2010). Viewing time effects revisited: Prolonged response latencies for sexually attractive targets under restricted task conditions. Archives of sexual behavior, 39, 1275-1288.

[MarcoViola88]

(messy) NOTES about 3 October meeting

• IDEAS for Study2:
  -- block design? (for REAL / ARTIFICIAL) [… with some reset due to attention check?] (DENIED)
  -- ... or: drop neutral pictures altogether? (APPROVED)
  -- go from 60 stimuli (only 20 arousing) to 50 (SORT-of-APPROVED)
  -- GOAL: make the effect stronger!

• DATABASE for Study2: we decide to use the same! (EroNAPS) (APPROVED)

ISSUES:

• ceiling effect / loss of sensibility ???
  -- … what about picking also some LOW arousing pictures? (APPROVED)
• enticement & arousal seem conceptually distinct, but they were largely overlapping
  -- we resolved to hypothesize a few phrasing and then to drop 'enticing' if we cannot make it sufficiently distinct

IMPROVEMENTS:

• no need to translate it, English only will do (APPROVED)
• more phone-friendly module for presentation, with 7-points likert (instead of analog) scales (APPROVED)
• some attention check (e.g., simply asking honestly "how attentive were you?" / "how much fun?") (APPROVED, in principle)
• more convincing cover-story (e.g., "we are collaborating with a start-up..." with a logo) but then we ask couple of questions to check whether they believe the instructions (e.g., "do you think labels are incorrect?") (APPROVED)
• phase 2: "well, actually the labels were randomly assigned. Now you tell us whether the images were real/fake!" [CHECK: consistency of the trace, i.e. our previous label? or maybe is it arousal-driven?] (APPROVED)
• including (rough) eye-tracking optionally, w/2 mins calibration (APPROVED)
  -- ... BUT: what about phones? also: can we make it incentivized/made mandatory on Prolific? (probably so!)

BEHAVIORAL DATA:

• what about taking RT? or a decision paradigm ("which image do you prefer?") (DENIED: a good idea ... for a follow-up study!)

TIMELINE for STUDY2:

• working paradigm within 20th Octorber, to be tested
• data collection on Prolific (budget: 1500/2000€!)
• data analysed for December 2024
• wrapping up and working to a paper by the beginning of 2025