# Finding Datasets # Overview There are several tools in ATLAS to help you find datasets and information associated to these datasets. The three major tools that we will cover here are: - [Run Query](https://atlas-runquery.cern.ch/) - [COMA](https://atlas-tagservices.cern.ch/tagservices/RunBrowser/index.html) - [AMI](https://ami.in2p3.fr/) We will get you to play around with each of these tools, and try to get you to find answers to certain questions. Please ask us for help if you can't find the answers\! # Run Query tool The Run Query tool provides detailed run-level information for the real data. It is particularly useful for detector and operation tasks, when you are trying to understand the LHC and ATLAS detector conditions and settings. You can use the search query either on the web interface or using the command line. On the web interface there are links to more expert pages for exact details on certain topics. ## Exercise: Get Data Summary plots Question: Use the [Run Query web interface](https://atlas-runquery.cern.ch/) to get the Data Summary Total Luminosity plots (vs day) for the entire 2018 13 TeV pp dataset and the 2017 13 TeV pp dataset. <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- To find the figures look in the top right corner of the main run query page (https://atlas-runquery.cern.ch/) in the grey band with white lettering, for the subject "Data Summary". Click on that link which will take you to a new page. To change the year or data-type click on the subject "Data Set" in the top right corner.   -------------------------------------------------------------- <br> </details> These type of figures are shown very regularly in a diverse set of meetings, so it's nice to understand how to find them\! ## Exercise: Find luminosity blocks with stable beams Use the [Run Query web interface](https://atlas-runquery.cern.ch/) to find which luminosity blocks had stable beams in Run **325558**. <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- In the Run Query search field use the query: `find run 325558 / show all` When the search is complete scroll down to see the run and all the information associated to the run. Under the main column "LHC and online luminosity information" you will see a sub-column called "Stable beams". Below that column there is a list of luminosity block (LB) ranges with the following information: ``` LB 1-115: FALSE LB 116-511 TRUE LB 512-603 FALSE ``` So the answer is luminosity blocks 116-511 were where stable beams were declared. **Bonus sub-question**: What is required for stable beams to be declared? -------------------------------------------------------------- </details> You may need to investigate and correlate trigger rates with LHC conditions, such as when stable beams were declared. ## Exercise: Find run details Use the [Run Query web interface](https://atlas-runquery.cern.ch/) to help you find the answers to the following questions about run **335302**: 1. What date was it taken? 2. What period was it taken in? 3. What was the prescale of the trigger `HLT_g60_loose` during luminosity blocks 257-268? <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- In the Run Query search field use the query: `find run 335302 / show all` a. 13 September 2017 b. Period G1 c. Under the column "Trigger information: Prescale key", click on the luminosity block range of interest which will take you to a new page. In that new page you will see the prescale for trigger `HLT_g60_loose` during that time was 1359.0. -------------------------------------------------------------- </details> # COMA COMA provides quick information about data in containers/periods/run/luminosity-blocks, and is a tool that aggregates information from across many runs. It is a useful tool to get an overview of the configurations of streams and triggers. To access the COMA web portal you need a valid grid certificate in your browser and you need to be a part of the ATLAS virtual organization (VO) [(tutorial instructions)](https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/). When trying to access the web portal accept any site warnings and click `OK` (not `CANCEL`) ## Exercise: Understanding data periods Question: Use the [COMA web portal](https://atlas-tagservices.cern.ch/tagservices/RunBrowser/index.html) to find out what were the data periods taken in 2017 for 13 TeV pp collisions? And what happened in period E (i.e. what defines period E)? <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- There are likely several ways to find this answer. My way is the following: - from the main page click on the "COMA Period Menu" - in the new page, under "By project", click on data17\_13TeV - in the new page, click on \[A:N\] in yellow under "All Year" You will see the listing of periods (and associated information) for periods A to N. In period E the Description column states that "Runs after the switch of the online release to 21.1.10". -------------------------------------------------------------- </details> ## Exercise: Determine lowest unprescaled trigger Question: Use the [COMA web portal](https://atlas-tagservices.cern.ch/tagservices/RunBrowser/index.html) to find the lowest pt HLT unprescaled single muon trigger throughout the 2017 pp run. What was the corresponding L1 input item? <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- From the main page click on the Report Menu. And fill in the following fields: - Project: data17\_13TeV - Chain/Item: HLT\_mu\* And click the `SUBMIT` button at the bottom left. Expand the "Distinct Triggers" yellow link. You'll have to scroll (or do a clever search) to find the answer. With an answer of: **HLT\_mu15**. The corresponding L1 input item was **L1\_MU11**. -------------------------------------------------------------- </details> # AMI and pyAMI [AMI web interface](https://ami.in2p3.fr/) is the dataset metadata catalogue. It is a "mediating" application, which means it correlates information from many sources. pyAMI is the python client for AMI. It is in the ATLAS release, and also available standalone on lxplus, or as a tar file for download to your laptop. [pyAMI](https://ami.in2p3.fr/pyAMI) documentation explains how to use pyAMI. ## Exercise: Making an account in AMI To use AMI you must have a valid grid certificate and be registered with the [ATLAS VO](https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/). If this is your first time accessing AMI through the web interface you will need to setup an account. If you have not already done so, follow the instructions described in [Software Tutorial: Registering with AMI](https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/ami/). ## Exercise: Finding data formats Question: Using the [AMI web interface](https://ami.in2p3.fr/) , can you find the dataset we have been using for the tutorial? <details> <summary>Reveal Answer</summary> <br> -------------------------------------------------------------- Our file is `mc16_13TeV.410470.PhPy8EG_A14_ttbar_hdamp258p75_nonallhad.deriv.DAOD_PHYS.e6337_s3126_r10201_p4172` - From the main AMI web page click on 'Dataset Browser', select mc16 in simulated data and then select some tags to find our dataset - For example, select 'Data Type' and use the wildcard `%DAOD_PHYS% - Then select dataset number and type the ttbar process we have been using %410470% - If you click 'View Selection' you should see quite a few datasets - Find the one that matches our `e6337_s3126_r10201_p4172` and there you go! -------------------------------------------------------------- </details>