Upgrading to PsyNet 10¶
Introduction¶
PsyNet 10 brings a host of new features. We are excited about what these new features bring, but they do necessitate a few changes to experiments implemented with earlier PsyNet versions. This guide is intended to help you with that upgrade process.
Accumulating answers¶
PsyNet supports Trials containing multiple Pages. If accumulate_answers
is set to True
,
PsyNet used to accumulate the answers from these Pages into a single list, for example
["yes", "no", "no"]
. Subsequently we realized that this behavior is quite dangerous for
data analysis, because it’s easy to forget which page generated which response.
In PsyNet 10, answers are instead accumulated into dictionaries, for example
{"color": "green", "hours": 23.5}
. The keys for the dictionaries come from the
Page labels.
When exporting data from such Experiments, PsyNet now automatically unpacks these answers into separate columns. This means you don’t have to worry about unpacking the dictionary representation yourself.
Action needed¶
Action is only needed if your experiment uses answer accumulation
(i.e. you have accumulate_answers = True
in one of your Trial classes).
If so:
Check any code that refers to Trial answers (e.g.
participant.answer
ortrial.answer
), and make any necessary updates for the new dictionary representation.Update your analysis code to account for the new export format; instead of unpacking the list, each element of the answer should already be present as different columns in your CSV file.
S3 and asset management¶
Previous PsyNet versions required experimenters to rely on Amazon S3 storage for managing media files. They were expected to use various functions to interact with S3 manually, doing things like managing S3 access permissions, creating buckets, uploading to S3, and so on. This made it difficult to generalize a particular experiment implementation to other storage back-ends, or to run such experiments without Internet access (e.g. in the context of field research).
PsyNet 10 incorporates a much more sophisticated approach to media management. There is a new
database-backed object hierarchy based on the Asset
class, where each
media file is represented as an Asset object that is linked to the database.
Different storage backends are then represented by different subclasses of the
AssetStorage
class.
Now when the experimenter manipulates media files, they do not have to worry about things like S3 permissions,
file naming, anonymization, linking response data to uploaded media files, and so on.
They can simply write things like asset.deposit()
and everything will be managed for them.
Switching between different storage back-ends (e.g. from S3 to local storage) can be achieved
just by changing a single line of code in the Experiment class.
Action needed¶
If your experiments have any explicit interaction with S3 (which normally means calling PsyNet
functions with s3
in the name), then this code will probably throw an error because those
PsyNet functions no longer exist. If you really want to keep this code the same for now,
you can get the source code for these old functions by going to the latest v9.x.x version of PsyNet,
and copy this into your Experiment file. However, it is recommend that you instead embrace the new
Asset management system when you upgrade your experiment.
The best way to move forward here is to first read the new Asset documentation chapter.
Once you have read this, look for the PsyNet demo that matches closest to your current situation.
This is likely to be the static_audio
paradigm, which covers both pre-generated assets and
assets recorded during the experiment.
Note: Audio Imitation Chain experiments should not need any upgrading, as far as I can tell.
Static versus Chain experiments¶
PsyNet 10 consolidates the underlying implementation for Static experiments and Chain experiments into a common code-base. As a result, Chain experiments can now access various features that were originally only available in Static experiments, such as blocked designs and stimulus pre-generation.
The former implementation of Static experiments was rather complicated. One had to implement so-called
StimulusSpecs and StimulusVersionSpecs, which PsyNet compiled under the hood into Stimulus
and Stimulus Version objects which were stored in the database.
PsyNet 10 massively streamlines this procedure.
There is now no longer such thing as a Stimulus or a Stimulus Version; one just uses Nodes instead.
Moreover, the way for predefining experiment structure is now homogenized between Static and Chain experiments.
Rather than defining Stimulus Sets (for Static experiments)
or defining balance_across_networks
constructs (for Chain experiments),
one now just provides a simple list of Nodes to the trial maker,
with these Nodes defining e.g. the initial set of stimuli or the starting states of the chain networks.
Action needed¶
Stimulus sets¶
Instead of passing a list of Stimulus Specs to the stimulus_set
argument of the Trial Maker,
you should now pass a list of Nodes to the nodes
argument of the Trial Maker.
See the static
demo for an example.
Stimulus versions¶
Experiments using Stimulus Versions need to be reorganized. The standard solution is to turn each Stimulus Version into a Node. This has subtle implications for the balancing; previously balancing only controlled the accumulation of Trials across Stimuli, not across Stimulus Versions. This shouldn’t matter much for most people.
Accessing stimuli within trials¶
Some people’s experiments access the stimulus
object within trial methods, for example:
def show_trial(self, experiment, participant):
the_rule = self.stimulus.definition["rule"]
...
In such cases you should replace stimulus
with node
:
def show_trial(self, experiment, participant):
the_rule = self.node.definition["rule"]
...
More generally, it’s a good idea to do a full-text search for stimulus
throughout your code base
to find cases where it ought to be replaced with node
.
Assets¶
Old PsyNet experiments that use Stimuli with media (e.g. audio files) need to be updated
to use the new PsyNet asset management system. The best way to do this is to read the new
Asset
documentation, and then explore the static_audio
demo to see how assets are managed there.
It should be rather straightforward to update your code to follow this model.
Trial Makers¶
The built-in arguments for Trial Makers have been updated slightly and pre-existing code is likely to throw an error. Don’t worry, the fixes are very minor.
Some changes involve renaming
stimulus
tonode
.Others involve replacing
num_
withn_
.There is a new argument called
expected_trials_per_participant
, which is different frommax_trials_per_participant
; the former is used for estimating experiment duration, whereas the latter is used as a rule for determining when the participant stops receiving trials from the Trial Maker.The old way of assigning participants to participant groups was to override the
choose_participant_group
method of the Trial Maker. The new way is to provide a function to the Trial Maker’schoose_participant_group argument
, a function which takes one argument (participant
) and returns the chosen participant group.
To find the up-to-date list of Trial Maker arguments, use the autocomplete function of your IDE,
or visit the documentation for ChainTrialMaker
or StaticTrialMaker
depending on what’s appropriate.
Initializing chain experiments¶
Previously Chain experiments would initialize their chains using code like the following:
class CustomNetwork(AudioGibbsNetwork):
...
def make_definition(self):
return {"target": self.balance_across_networks(TARGETS)}
In PsyNet 10, networks are instead initialized by passing an optional list of Nodes
to the start_nodes
argument of the Trial Maker. For example:
start_nodes=lambda: [CustomNode(context={"target": target}) for target in TARGETS],
This new approach is much more flexible, and moreover allows the experimenter to provide assets for initializing those chains.
Custom network classes¶
Implementing Chain experiments used to involve implementing custom Network classes, for example:
class CustomNetwork(AudioGibbsNetwork):
synth_function_location = {
"module_name": "custom_synth",
"function_name": "synth_stimulus",
}
s3_bucket = "audio-gibbs-demo"
vector_length = DIMENSIONS
vector_ranges = [RANGE for _ in range(DIMENSIONS)]
granularity = GRANULARITY
n_jobs = 8 # <--- Parallelizes stimulus synthesis into 8 parallel processes at each worker node
def make_definition(self):
return {"target": self.balance_across_networks(TARGETS)}
This process has now been streamlined to avoid the need to define a custom Network class. Instead all relevant parameters have been migrated to the custom Node class. For example:
class CustomNode(AudioGibbsNode):
vector_length = DIMENSIONS
vector_ranges = [RANGE for _ in range(DIMENSIONS)]
granularity = GRANULARITY
n_jobs = 8 # <--- Parallelizes stimulus synthesis into 8 parallel processes at each worker node
def synth_function(self, vector, output_path):
custom_synth.synth_stimulus(vector, output_path)
Note that in this particular case (Audio Gibbs) there are several other changes too that have streamlined the definition of the Custom Node class. They’re covered in other parts of this documentation.
Action needed¶
If you have a Chain experiment you will need to migrate most elements from your custom Network class to your custom Node class. The precise migration required depends on which paradigm you are using. Look at the corresponding PsyNet demo for guidance here.
Audio Gibbs experiments¶
In addition to the changes noted above, the Audio Gibbs pattern now has a simplified mechanism for specifying the synthesis function. Instead of this Network attribute:
class CustomNetwork(AudioGibbsNetwork):
synth_function_location = {
"module_name": "custom_synth",
"function_name": "synth_stimulus",
}
We now have this Node attribute:
class CustomNode(AudioGibbsNode):
def synth_function(self, vector, output_path):
custom_synth.synth_stimulus(vector, output_path)
Action needed¶
If you have an Audio Gibbs experiment you need to update your synthesis function specfication to match the pattern described above.
Sources¶
Former PsyNet versions had the concept of Sources. Sources were used as the starting point for chains in paradigms such as Serial Reproduction and Gibbs Sampling with People. We have now streamlined the syntax for such experiments and eliminated the need for Sources, subsuming their function under the Node class.
Action needed¶
This change should not impact most people’s Experiment code. It may impact your analysis code, depending on how it is implemented, but quite possibly not.
prepare_trial
¶
There is a Trial Maker method called prepare_trial
which is responsible for preparing the
next trial that the participant receives. Originally this method was expected to return
either a Trial object or None
, with the latter signifying that the Trial Maker should terminate.
The signature of this method has now changed; it’s now expected to return a tuple where the first
element is the Trial object, as before, with None
if no Trial is found, and the second element
being a string taking one of three values: “available”, “wait”, and “exit”.
Most experiments do not touch the prepare_trial
method. However, experiments that do override it
need to be updated for PsyNet 10. For example, one’s original code might look like this:
def prepare_trial(self, experiment, participant):
if participant.var.has("expire"): # finish the game
logger.info("Ending game")
return None
return super().prepare_trial(experiment, participant)
Such code should be updated to this:
def prepare_trial(self, experiment, participant):
if participant.var.has("expire"): # finish the game
logger.info("Ending game")
return None, "exit"
return super().prepare_trial(experiment, participant)
Accessing trials¶
Previously it was possible to access an object’s trials by writing
network.trials
, node.trials
, or participant.trials
.
We have moved on from this nomenclature because (partly for historic reasons)
it was not always clear whether the returned list included failed trials or not.
These attributes have now been replaced with the following:
.all_trials
- returns all trials owned by the object;.alive_trials
- returns all non-failed trials owned by the object;.failed_trials
- returns all failed trials owned by the object.
Action needed¶
Replace all occurrences of .trials
with one of the three attributes listed above.