Data Formats

Collecting Snapshots and Data Sets via the Blu-Ice interface

‘Snapshots’: In the Blu-Ice user interface, there is a designated “run” labelled as ‘0’. This is set up to simply capture snapshots of crystal diffraction patterns in order to test a crystal’s diffraction capability and plan strategies for data collection.

‘Data Sets’: Apart from run ‘0’, all subsequent run numbers can be used for collecting data sets. This is achieved by clicking on the ‘*’ tab (new run tab) and populating the fields related to each collection parameter. Thereafter, users can choose to re-use any run to collect further data sets without having to re-enter the same collection parameters. The run settings will be retained even when crystals are exchanged or the hutched is opened; runs are only lost when a user chooses to delete the run from their sequence.
Note: CARE should be taken when re-using “run” tabs. The image name, or at least the destination directory, should be changed to signify a new data set. If users fail to individually name each data set, the Blu-Ice software will automatically generate a subdirectory called OVERWRITTEN_FILES, where the older of the ‘duplicate’ files will be stored. This will protect the images from accidental overwrite, but will mean that the second data set may potentially be mis-labelled. Once the existing run is finished, the software will go on to collect any unfinished runs following the existing one.

File Naming Conventions
The ADSC Quantum Detectors produce image files with an extension of .img

In Blu-Ice, each image file within a given run sequence is named in the following way:

‘file prefix’‘run number’‘energy number’_‘image number’.img

For example, a file related to the 10th frame collected in Run 2 at the 2nd input energy might be named:

data_2_E2_010.img

With only one energy level selected, the file would be named:

‘file prefix’‘run number’‘image number’.img

The run sequence depends on the chosen values for phi, wedge, energy or inverse beam. The image collection order will be displayed in the ‘run sequence’ window throughout data collection.

Additional Information related to Fluorescence Scan Outputs
The following files are written out after successful completion of a fluorescence scan:

• name-scan: Contains the raw fluorescence readings
• name-scansmooth_exp.bip and namescansmooth_norm.bip: Intermediate files from autochooch
• name-fp_fpp.bip: Anomalous scattering factors (f” and f’) calculated by autochooch
• name-summary: Summary of the scan, including the values for the peak, inflection and remote wavelengths and the f” and f’ values for each of them.

Greater detail about Data Formats can be located in the User Manual (Blu-Ice section), which can be downloaded from the Resources webpage for this beamline.

Data Processing

At the beamline we have available XDS and iMosflm for data reduction. We also have available pipelines like XDSme and Xia2.

CIF fields specific to the MX1/MX2 Beamlines

Please see the text file below for information on what values/wording to use for CIF fields specific to the MX1 and MX2 Beamlines at the Australian Synchrotron:

CIF information

Data Transfer and Retention

Data will be stored at the Australian Synchrotron for 90 days after the users’ allocated beamtime.

It is the User’s responsibility to ensure that data sets are retrieved and adequately backed-up DURING their beamtime. It is highly advisable that users connect their own hard drives, and check on data back-up, during usual business hours (i.e. whilst IT support is available), as staff will not return to site to assist with data storage issues after-hours.

Ask staff about the 'HDsync' script. This backs up your data as you are collecting.

HDsync /media/##### XXXX            ##### = your hard drive name (please ensure that there are no spaces in the name), XXXX = your experiment number (provided to you by staff)

Due to the high turnover of user groups on the Macromolecular Crystallography beamlines (formerly known as the Protein Crystallography beamlines), groups will not be permitted to stay beyond their allocated shifts to transfer their data. Users should also note that when a ‘Run’, or Synchrotron calendar cycle, is finished (i.e. 3 times annually) all data access will be restricted to the BOSS system only. This allows staff to make arrangements for the next block of beamtime on the data server.

How do I retrieve my data?

External Hard Drive:

Users can bring/send an external hard drive, noting that we support VFAT or EXT3 file systems ONLY. It is highly recommended that user groups allocate a minimum of 500GB per 24 hour period. Remote access users who choose to send storage devices to the Australian Synchrotron should include return postage details and employ packaging that gives adequate protection for their devices during transit.

Electronic Data Transfer:

Users can make arrangements to access the Virtual BeamLine (VBL) system to transfer data electronically. Macromolecular Crystallography support staff will arrange user access to the data storage gateway (via the BOSS system) for ONE designated person per user group. This person must be nominated by the user group spokesperson PRIOR to their arrival at the Synchrotron so that the appropriate folders and permissions can be set up on the data server.

1) Once access to BOSS has been set up, users can go to:

http://vbl.synchrotron.org.au/

and follow the online instructions to use the Storage Gateway. Users will also need to know their Experiment Proposal Number to access the relevant data.