SCT Off-Detector Electronics Requirements Document Draft 0.1 September 20, 1996 Requirements assembled by: A.J. Lankford, University of California, Irvine, R.C. Jared, University of Wisconsin and LBNL, T. Weidberg, Oxford University, M. Postranecky, University College London and others Abstract This document is an incomplete draft version of the requirements upon the SCT off-detector electronics. This early draft is made available for comments and contributions by the SCT community. Please direct comments to offdet@lankford.ps.uci.edu. Editors Note This draft is nowhere complete and some sections are quite sketchy. These sections will get filled in as time goes on. We would welcome contributions to fleshing out of the draft and completing the requirements, as well as questions and comments concerning the requirements already included. The format which we have adopted for the requirements in this draft includes some mandatory fields. These are: name, requirement, justification, and status. Name is a short name by which the requirement can be refered. Requirement is a complete and clear but concise statement of the requirement. Justification states the basis for the requirement. It is an important field because it states the basis for imposing the requirement; therefore, it contains the history of the thinking behind the requirement. It also documents the assumptions that went into the requirement; hence, it states the assumptions that may be questioned should a designer find that a given requirement is difficult or impossible to meet. Status states the current status or completeness of the requirement. It is also important to the interpretation and possible questionning of a given requirement. For instance, a requirement which contains approximate values of parameters should not be misunderstood as containing exact or final values. This format also contains some optional fields (explanation, notes, comments) I would say that an explanation explains the requirement, in case the concise statement in the requirements field might not be clear by itself. I would say that notes contains references or additional information concerning a requirement. And I would say that comments are generally of a temporary nature and can be somewhat editorial. Text in square brackets are intended to be very temporary comments, reminders, or editorial notes. The format which we have adopted here may be changed for a future draft. For instance, we might consider changing to the ESO standard being used by some of the T/DAQ group. Table of Contents I Introduction II Brief Description of SCT III Overview of Off-Detector Electronics IV Essential Requirements IV.A Requirements regarding detector-mounted electronics IV.A.1 Receipt of data IV.A.1.a All data types IV.A.1.b Event data IV.A.1.c Calibration data IV.A.1.d Register data IV.A.2 Transmission of clock and control IV.A.2.a General IV.A.2.b Transmission of clock IV.A.2.c Transmission of commands IV.A.2.d Clock & Control procedures IV.A.2.e Additional requirements IV.A.3 Services IV.A.3.a Detector bias IV.A.3.b Electronics power IV.A.3.c Monitoring and control IV.A.3.d Detector protection IV.B Requirements regarding Trigger/DAQ IV.B.1 Readout Buffers (ROBs) IV.B.1.a Transmission of data to ROBs IV.B.1.b Flow control between RODs and ROBs IV.B.1.c Mapping of modularity onto ROBs IV.B.2 Timing, Trigger, and Control (TTC) System IV.B.2.a Functionality required by TTC IV.B.2.b Interface with TTC IV.B.2.c Signals from TTC IV.B.2.d Flow control to LVL1 IV.B.3 Data Acquisition System (DAQ) IV.B.3.a Configuration IV.B.3.b Monitoring of data flow IV.B.4 Detector Control System (DCS) IV.B.5 Additional Trigger/DAQ requirements IV.B.5.a Partitioning requirements IV.B.5.b Data flow control and synchronization requirements V Functional Requirements V.A Requirements regarding the Readout Crate V.A.1 Readout Driver V.A.2 TTC Distribution Module V.A.3 CPU module V.A.4 Crate and backplane V.B Requirements regarding the SCT TTC subsystem V.C Requirements regarding the SCT DAQ subsystem V.D Requirements regarding the SCT DCS subsystem V.E Requirements regarding the SCT-specific functions in the ROBs servicing the SCT VI Glossary I Introduction [introduces the purpose of this document] [The SCT off-detector electronics includes all electronics associated with the SCT which is not mounted on the detector with the exception of: .the system of power supplies and other electronics which provide DC services to the SCT, including their control system and cables, .the system for environmental monitoring and control of the SCT, .the clock and control links and data links connecting the off-detector and detector-mounted electronics, including the drivers and receivers at the off-detector end of the links.] II Brief Description of SCT [this section will contain text which briefly describes the purpose, layout, and data bandwidths of the SCT III Overview of Off-Detector Electronics [this section will contain text which introduces the functionality and likely physical layout of the off-detector electronics IV Essential Requirements IV.A Requirements regarding detector-mounted electronics IV.A.1 Receipt of data The off-detector electronics receives data from the on-detector electronics. The data received can be any of three different types: event data, calibration data, or register data. The following requirements pertain to receipt of data of any of the three types. The interface between the off-detector electronics and the detector-mounted electronics is defined to be at the output of data link receiver circuitry which is to be provided by SCT Data Transmission group. For instance, if the data link is optical, then the link receiver circuitry does opto-electrical conversion, outputting a serial data stream. IV.A.1.a All data types The following requirements apply to the receipt of data of any data type. Source of input data streams 1) The off-detector electronics must receive input data streams from the detector-mounted electronics. Justification: Receipt of data from the detector-mounted electronics is one of the primary missions of the off-detector electronics. Status: Firm. Number of input data streams per SCT detector module 2) In the SCT, two input data streams must be received from each detector module. Justification: Each detector module is instrumented with two output data streams in order to offer a degree of redundancy in case of failure of one data stream. Explanation: Each detector module is instrumented with two output data streams. During normal operation, each data stream reads out the data from one-half of the channels on the module. If one data stream fails, the detector-mounted electronics can be reconfigured such that the remaining operational data stream reads out all of the channels on the module, however, at reduced aggregate bandwidth. Status: Firm. Total number of input data streams 3) In the SCT, the total number of input data streams is 8176. Justification: The total number of SCT detector modules is 4088. Since two input data streams exist per detector module, 8176 data streams exist in total. Status: Incomplete and not firm. The total number of input data streams xxx and the number of SCT detector modules must be filled in. The number of SCT detector modules depends upon the layout of the SCT. The current layout has been stable for a some months and is considered to be a baseline design. Nonetheless, the number of SCT detector modules may change by as much as 10-20% in the future. Masking of input data streams - 1 4) The off-detector electronics must be capable of being configured under program control so as to eliminate data from any specified set of input data streams. All data on the specified set of data streams should be eliminated from the time that the off-detector electronics is so configured until it is reconfigured to accept data on those input streams. Justification: Detector-mounted electronics may fail in a mode which produces meaningless or excessive data on some of the input data streams to the off-detector electronics. The off-detector electronics must be able to eliminate such data in order to not overly burden the data bandwidth capabilities of subsequent portions of the readout chain. It is necessary to have this functionality in the off-detector electronics because the failure of the detector-mounted electronics may not allow disabling of the failed elements directly. Notes: Only two states, accept data or eliminate data, are required for each input data stream. In the eliminate data state, all data is eliminated (i.e. data is not eliminated selectively). In the accept data state, all data is accepted if it meets all other acceptance criteria. In order to avoid undue complexity in the off-detector electronics, states in which some data is eliminated and some is accepted according to some criteria such as the type of data are not required (unless explicitly demanded by some other requirement). There may be circumstances (as demanded by some other requirement) that result in certain data records on some input data streams to be eliminated. For instance, a data record with an identified error in its header may be eliminated. Under these circumstances, a flag will be inserted into the output data stream from the off-detector electronics. When the off-detector electronics is configured according the requirement stated here, it is not required to insert a flag into the output data stream. Once configured to eliminate data on a specified set of data streams, the off-detector electronics should continue to eliminate all data on that set of streams in order that the response of the off-detector electronics to data on those streams is well-defined. Status: Firm. Masking of input data streams - 2 5) The off-detector electronics must be capable of being configured under program control such that it can operate normally if no signal is received on any one of a specified set of input data streams. Justification: Detector-mounted electronics may fail in a mode which provides not data on some input data streams to the off-detector electronics. In addition, some data stream inputs to the off-detector electronics may not be connected to detector-mounted electronics (i.e. some inputs may not be utilized). The off-detector electronics must be capable of receiving and processing data on all other data streams normally under these circumstances. Comment by AJL: I am not certain that this requirement needs to be implemented separately from Masking of input data streams - 1. Status: Firm. Input data types 6) The off-detector electronics must be capable of receiving data of three data types (event data, calibration data, and register data) on each input data stream. Justification: The detector-mounted electronics produces these three data types. Consequently, the off-detector electronics must be capable of receiving all three types. Notes: The processing required of the off-detector electronics in response to receipt of each data type is elaborated in subsequent requirements. At this time, it is assumed that a fourth data type for monitor data is not required. Explanation: Some text here should explain what these three data types are and to what commands they are responses. Status: The requirement to receive these three data types is firm. It is not yet out of the question that additional data types will be required as well. Media of input data streams 7) The off-detector electronics must be capable of receiving input data streams from receiver circuitry provided by the SCT Data Transmission group. The characteristics of the input signals will be specified in the interface specifications document for the SCT data links. Justification: Notes: Further summary of signal characteristics would be useful here. Status: Baud rate of input data streams 8) The off-detector electronics must be capable of receiving data transmitted at 40Mbs (i.e. 40 Mbits/s) on the input data streams. Format of input data 9) The off-detector electronics must be capable of receiving data on the input data streams which is formatted according to the protocol defined in the specifications of the on-detector electronics. Justification: Notes: [Provide a description of the data format here; however, refer to the appropriate document for the specifications.] Status: Flow control of input data streams 10) The off-detector electronics must be capable of receiving data on the input data streams without any direct flow control mechanism. Justification: Explanation: [include an explanation of what is meant by no direct flow control mechanism, i.e. that the detector-mounted electronics pushes the data whenever it feels like it.] Status: Negligible data loss 11) The off-detector electronics must be capable of receiving all data on the input data streams without loss of any data, except in very unusual circumstances. Justification: Explanation: [explain here why it is always the case that data may be lost] Comment: We should try to quantify very unusual circumstances. A reasonable criteria might be significantly less data loss than results from inadequate buffering in the detector-mounted electronics. Status: Data loss 12) In the case of data loss during very unusual circumstances, the loss of data must be flagged in the output data stream. Justification: Note: [describe the proposed method of flagging data loss] [We may choose to require the counting the number of times data is lost on each input data stream.] Status: Recognition of data type 13) The off-detector electronics must interpret the data protocol of the input data streams in order to recognize the data type and in order to respond appropriately to the data type received. Justification: Comment: We should probably clarify what is meant by respond appropriately in either the requirement or an explanation field. Status: Protocol errors 14) The off-detector electronics must recognize protocol errors in the input data stream, and it must flag the occurrence in the output data stream. Justification: Status: Recognition of error flags 15) The off-detector electronics must recognize error flags in the incoming data stream. Justification: Notes: [include a description of possible error flags] Status: Monitoring of errors in input data stream 16) A local CPU in the off-detector electronics must monitor the occurrence and frequency in each data stream of protocol errors and of error flags. Justification: Status: Synchronization of input data streams 17) The off-detector electronics must provide a mechanism which adjusts the phase of data on each input data stream relative to the clock which operates the off-detector electronics. Justification: Status: Disabling of error checking 18) The off-detector electronics must be capable of being configured by program control to disable error checking on any set of input data streams. Justification: Status: IV.A.1.b Event data Event data generally consists of data records associated with physics triggers. More specifically, event data consists of data records associated with L1Accept commands but which are not associated with CalStrobe commands. The following requirements pertain to the processing of event data but not to both the processing of calibration data and of register data. Merge event data streams 1) The off-detector electronics must merge event data records from multiple input data streams into output event data records on output data streams. Justification: Comment: If there is a requirement upon the number of input data streams merged into a single ouput data stream, then that number should be included in the requirement defined here. Status: Buffering of event data 2) The off-detector electronics must buffer output event data records for transmission to Readout Buffers. Justification: Status: No loss of buffered event data 3) The off-detector electronics must not overwrite or otherwise lose output event data records before they are transmitted to Readout Buffers. Justification: Comment: This requirement as written dictates that data is not overwritten, implying that data which would cause overflow is dropped. An alternative to dropping data from new events is to overwrite old events (as is done in the ABC). Status: Local access to event data records 4) The off-detector electronics must buffer output event data records for access by a local CPU for monitoring purposes. Justification: Status: Monitoring of event data 5) The off-detector electronics must provide local computing to monitor output event data records in order to monitor detector performance. Justification: Explanation: [explain local, computing, monitor] Status: Communication of monitoring results 6) Local CPUs provided to monitor output event data records must report the monitor results to the SCT subsystem of the DAQ control system. Justification: Status: IV.A.1.c Calibration data Calibration data consists of data records associated with L1A commands which are preceded immediately by CalStrobe commands. The following requirements pertain to the processing of calibration data but not to both the processing of event data and of register data. Merge calibration data streams 1) The off-detector electronics must merge calibration data records from multiple input data streams into output calibration data records. Justification: Status: Local access to calibration data records 2) The off-detector electronics must buffer output calibration data records for access by local computing for monitoring purposes. Justification: Status: No loss of buffered calibration data 3) The off-detector electronics must not overwrite or otherwise lose output calibration data records before they are accessed by a local CPU, unless calibration data loss is enabled under program control. Justification: Status: Processing of calibration data 4) The off-detector electronics must provide computing to process calibration data, store calibration data locally for control purposes, and report summaries of calibration results to the SCT subsystem of the DAQ control system. Justification: Note: Some examples of local use of calibration data for control purposes are identifying channels that should be masked and determining settings of threshold voltages. Status: IV.A.1.d Register data Register data consists of data records associated with commands which operate on registers in the detector-mounted electronics. The following requirements pertain to the processing of register data but not to both the processing of event data and of calibration data. Merge register data streams 1) The off-detector electronics must be capable of merging register data records from multiple input data streams into output register data records when the detector-mounted electronics responds to commands by simultaneously producing register data on multiple input data streams. Justification: Explanation: For instance, in response to a broadcast command to read a register in each front-end chip of each module. Comment: Do we have such commands? Status: Respond to a single register data stream 2) The off-detector electronics must be capable of receiving register data on a single input data stream under circumstances in which not all other input data streams are disabled. Justification: Some commands to the detector-mounted electronics address a register in a single detector-mounted IC, producing data on a single input data stream to the off-detector electronics. Status: text Local access to register data records 3) The off-detector electronics must buffer output register data records for access by local CPUs. Justification: Status: No loss of buffered register data 4) The off-detector electronics must not overwrite or otherwise lose output register data records before they are accessed by a local CPU, unless register data loss is enabled under program control by the local CPU. Justification: Status: Processing of register data 5) The off-detector electronics must provide local CPUs to process register data, store register data locally for configuration and control purposes, and report summaries of the results of configuration and control procedures to the SCT subsystem of the DAQ control system. Justification: Status: IV.A.2 Transmission of clock and control IV.A.2.a General The off-detector electronics transmits the 40MHz bunch crossing clock and all control commands to the detector-mounted electronics on 40Mbps serial data links. These links are presently planned to be fiber optic; however, twisted pairs are also being considered. If the links are fiber optic, the clock signal and control commands to each detector module share the same link. If the links are twisted pair, then the clock signal and control commands are likely to be transmitted on separate links. The interface of the off-detector to the data transmission is defined to be at the input to link driver circuitry which is to be provided by the SCT data transmission group. For purposes of this document, it is assumed that the input to the link driver is the 40MHz bunch crossing clock and a 40Mbps serial command stream. The link driver circuitry processes these input signals and outputs the optical or electrical signal or signals. The following requirements pertain to transmission of clock and control. Clock and control transmission 1) The off-detector electronics must transmit clock signals and control commands to the detector-mounted electronics. Justification: Status: Clock and control stream 2) The off-detector electronics must be capable of transmitting both clock signals and control commands such that the clock signal qualifies (i.e. strobes) the signal of the control commands at the receiver circuitry in the detector-mounted electronics. Justification: Note: Together the clock signal and the control command signal form what is refered to as a clock and control stream (or Clk&Ctl stream) Status: Number of output Clk&Ctl streams per SCT detector module 3) In the SCT, two output clock and control streams must be transmitted to each pair of detector modules. Justification: Each detector module is instrumented with one input Clk&Ctl stream; however, the two Clk&Ctl streams which instrument a pair of detector modules are shared by the two modules to provide a degree of redundancy in case of failure of one of the streams. Explanation: Each detector module is instrumented with one input Clk&Ctl stream. The Clk&Ctl logic of each pair of detector modules is interconnected. During normal operation, each Clk&Ctl stream transmits to a single detector module. If one Clk&Ctl stream to a pair of modules fails, the detector-mounted electronics can be reconfigured such that the remaining operational Clk&Ctl stream transmits to both modules in the pair. Comment: This requirement needs to be rewritten in order to properly reflect the use of Clk&Ctl in the detector-mounted electronics. A requirement to organize the outputs in a fashion which facilitates the detector-mounted electronics is needed. Status: Firm. Total number of output Clk&Ctl streams 4) In the SCT, the total number of output clock and control streams is 4088. Justification: The total number of SCT detector modules is 4088. Since there is one output Clk&Ctl stream per detector module, there are also 4088 Clk&Ctl streams in total. Status: Established, but subject to possible change. The number of SCT detector modules depends upon the layout of the SCT. The current layout has been stable for a some months and is considered to be a baseline design. Nonetheless, the number of SCT detector modules may change by as much as 10-20% in the future. Masking of output Clk&Ctl streams 5) The off-detector electronics must be capable of being configured under program control so as to not transmit clock signals or control commands to any specified set of output clock and control streams. No transmission on the specified set of clock and control streams should exist from the time that the off-detector electronics is so configured until it is reconfigured to transmit clock and control on those output streams. Justification: Status: Media of output Clk&Ctl streams 6) The off-detector electronics must be capable of delivering a clock signals and serial command streams to transmitter circuitry provided by the SCT data transmission group. The characteristics of the clock signals and command stream will be specified in the interface specifications document for the SCT data links. Justification: Notes: Further summary of signal characteristics would be useful here. Status: Frequency of output clock signal 9) The off-detector electronics must be capable of transmitting a 40MHz clock signal on the output clock and control stream. Justification: Status: Baud rate of output control stream 10) The off-detector electronics must be capable of transmitting control commands at 40Mbs (i.e. 40 Mbits/s) on the output clock and control streams. Justification: Status: Protocol of Clk&Ctl stream 11) The off-detector electronics must be capable of transmitting clock signals and control commands on the output clock and control streams to the protocol defined in the specifications of the detector-mounted electronics. Justification: Notes: [Provide a description of the clock and control protocol here; however, refer to the appropriate document for the specifications.] Status: Flow control of output Clk&Ctl streams 12) The off-detector electronics must transmit control commands on the output clock and control streams to the detector-mounted electronics without any flow control mechanism in the detector-mounted electronics. Justification: Explanation: [include an explanation of what is meant by no flow control mechanism] Status: IV.A.2.b Transmission of clock Clock signal characteristics 1) The clock signal which the off-detector electronics transmits to the detector-mounted electronics must be a 40MHz derived from the bunch crossing clock delivered by the TTC system Justification: Status: Clock phase delay 2) The phase delay of each clock signal transmitted by the off-detector electronics to the detector-mounted electronics must be programmable in steps of 1 ns over a range of 25 ns. Justification: [adjust relationship of clock to detector signals] Status: Clock stability 3) The long-term stability (i.e. of order one day) of each clock signal transmitted from the off-detector electronics to the detector-mounted electronics must be better than 1 ns. Justification: Status: Clock jitter 3) The short-term jitter of each clock signal transmitted from the off-detector electronics to the detector-mounted electronics must be less than +/- 0.5ns RMS. Justification: Status: IV.A.2.c Transmission of commands Control encoded with clock 1) Serial control commands delivered by the off-detector electronics to the clock and control transmitter circuitry must be qualified with the clock signal according to the Interface Specification to the Detector-Mounted Electronics. Justification: Status: Names of commands 2) The off-detector electronics must be able to transmit both prompt commands, such as L1A, and non-prompt commands, such as, Write Calibration Mask. Justification: Status: Command syntax 4) The syntax of commands, that is the meaning of the bits in the serial command stream, transmitted by the off-detector electronics to the detector-mounted electronics must conform to the syntax described in the Specification of the ATLAS Binary Chip (ABC). Justification: Status: Origin of commands 5) The off-detector electronics must be capable of transmitting commands which are issued locally, within the off-detector electronics, or which are received from either the TTC or DAQ systems. Justification: Status: Protocol of commands 6) The protocol of commands (i.e. the amplitude and timing characteristics of the signals which represent the bits in the serial command stream) of the commands from the off-detector electronics to the detector-mounted electronics must conform to the specifications in the Interface Specification to the Detector-Mounted Electronics. Justification: Status: Translation of commands 7) The off-detector electronics must translate TTC and DAQ commands from the TTC and DAQ syntax and protocol to the SCT syntax and protocol. Justification: Status: Addresses of commands 8) The off-detector electronics must direct TTC and DAQ commands to specific detector modules according to addresses received from the TTC and DAQ systems. Justification: Status: IV.A.2.d Clock & Control procedures [This section should include requirements associated with procedures, such as synchronization procedures or initialization procedures, for the detector-mounted electronics. It should also include a description of each procedure (or reference to another document containing a description.] [We probably have some requirements on having ability to verify the correct response on the part of the detector-mounted electronics to commands.] [What are the requirements upon when various commands can be issued? Can undefined command codes be issued? What are response times to various commands? What are requirements upon how promptly a command must be issued.?] IV.A.2.e Additional requirements [We probably have some requirements on having ability to verify the correct response on the part of the detector-mounted electronics to commands.] [What are the requirements upon when various commands can be issued? Can undefined command codes be issued? What are response times to various commands? What are requirements upon how promptly a command must be issued.?] IV.A.3 Services [This is the correct place to include any requirements upon off-detector electronics relating to services in the detector-mounted electronics. If there are no requirements upon the off-detector electronics, then this section can be deleted.] IV.A.3.a Detector bias IV.A.3.b Electronics power IV.A.3.c Monitoring and control IV.A.3.d Detector protection IV.B Requirements regarding Trigger/DAQ IV.B.1 Readout Buffers (ROBs) IV.B.1.a Transmission of data to ROBs Readout Links 1) SCT Readout Drivers must transmit output event records to standard-ATLAS Readout Buffers on standard-ATLAS Readout Links, as specified in ATLAS Readout Link Specification Document. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.1. Notes: The specification of the RO link will be provided by the T/DAQ group. The SCT group is responsible for providing the links. The T/DAQ group will provide the Readout Buffers (ROBs). Rule 5.3 of the Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0 also states that the output port of the Readout Link must comply with the ROB specification. Status: Well-established and stable; however, the Readout Link specification is not yet complete. Readout rate 2) SCT Readout Drivers must transmit data on Readout Links at the L1A event rate. The Readout Drivers must be capable of transmitting data at L1A event rates as high as 100kHz. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.2. Status: Well-established and stable. Event order 3) SCT Readout Drivers must transmit event records on Readout Links in event order. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 4.5 in order to facilitate event handling and checking in the ROB. In particular, it helps to avoid a time out mechanism in the ROB for missing events. Explanation: The expression event order means in monotonically increasing and continuous order of increasing ROD_L1ID. Note: This requirement implies that Readout Drivers must insert output an event record in case of a missing ROD_L1ID in the monotonically increasing order. Status: Well-established and stable. Readout Link bandwidth - 1 4) The data bandwidth of each SCT Readout Link must be large enough to accommodate the average data rate expected on that link for the maximum LHC luminosity and a L1A rate of 100kHz. Sufficient safety margin must be included for uncertainties in the link occupancy and accounting for protocol overhead. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.2. Note: This requirement imposes constraints on the grouping of input data links onto output data links. These constraints should be reflected in further requirements. Status: Well-established and stable. Readout Link bandwidth - 2 5) If it is foreseen that the Readout Driver will transmit calibration data to the Readout Buffer over the Readout Link, then the data bandwidth of each SCT Readout Link must be large enough to accommodate this potentially large data rate. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.4. Note: It is not presently foreseen to send large amounts of SCT calibration data to ROBs via SCT Readout Links. The path foreseen for calibration data is via the local CPU in the ROD crate and the SCT DAQ LAN. Status: Well-established and stable. Readout Link protocol 6) SCT Readout Drivers must transmit data on Readout Links according to the protocol specified in the Readout Link Specification Document. Justification: Status: Readout Link data format 7) The format of output event records must conform to the standards specified in the Readout Buffer Interface Specifications document. Justification: Notes: [The following description is out of date.] Notes: The format of output event records is currently defined as: 33-bit start of block control word block of 32-bit words 33-bit end of block control word. The block of 32-bit words contains: variable length header variable length data. At present the header consists: (1) format version, (2) a reserved word, (3) ROD_L1ID, (4) ROD_BCID, (5) size of a status block, (6) size of variable length data, (7) a variable length status block, (8) event type, (9) trigger type. The first words of the variable length data are an SCT-specific header. The start of block and end of block words contain a 33rd bit which flags these as control words. The start of block word contains a type number of the block. The end of block word contains an error summary bit. Status: Readout Link test mode 8) The Readout Drivers must provide a diagnostic mode in order to test the Readout Links in situ. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.2. Notes: This requirement needs elaboration in order to define the required tests. The specification of these tests needs to be worked out in conjunction with the ROB design. Status: Well-established and stable; however, this requirement needs elaboration. IV.B.1.b Flow control between RODs and ROBs Response to XOFF - 1 1) SCT Readout Drivers must suspend transmission on a Readout Link from a specific Readout Driver to its associated Readout Buffer when an XOFF bit from the Readout Buffer is received by the Readout Driver. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.2. Status: Well established; however, this requirement may need to be eleborated when the Readout Link protocol is finalized. Response to XOFF - 2 2) Each Readout Driver must assert a ROD_BUSY signal if its buffers are full and it is unable to transmit data to its Readout Buffer. [elaborate] [This requirement probably belongs in the section on TTC.] Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 5.2. Status: text IV.B.1.c Mapping of modularity onto ROBs RoI mapping 1) The off-detector electronics must map the geometry of the SCT onto the Readout Buffers associated with the SCT in a fashion that facilitates the access and processing of Regions of Interest (RoIs) within the SCT by the LVL2 Trigger. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 4.1. Status: IV.B.2 Timing, Trigger, and Control (TTC) System IV.B.2.a Functionality required by TTC The following requirements concern functionality of the off-detector electronics which is dictated by the ATLAS TTC system. Most of these requirements are dictated by the basic requirement that the off-detector electronics extend the functionality of the TTC system to the detector-mounted electronics. SCT TTC subsystem 1) The off-detector electronics must provide an SCT TTC subsystem which extends to the detector-mounted electronics the functionality provided by the ATLAS TTC system which is required by the detector-mounted electronics. Justification: implied by T/DAQ interfaces document Note: The functionality provided by the ATLAS TTC system which is required by the detector-mounted electronics is detailed in other requirements. Together the SCT standard TTC system and the SCT-specific extension of the standard TTC system comprise the SCT TTC subsystem. Status: Fundamental. Well established. SCT standard TTC system 2) The SCT TTC subsystem must provide a standard TTC system as defined in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 7.1. Justification: Note: The standard TTC system will be provided by the T/DAQ group. (See Rule 7.1.) Status: SCT-specific TTC extension 3) The off-detector electronics must provide an SCT-specific extension of the standard TTC system. Justification: The SCT-specific extension is required by the preceding requirement for a standard TTC system and by the interfaces to the detector-mounted electronics implied by the specification of the readout ICs. Note: The SCT-specific extension must be provided by the SCT group. (See Rule 7.4 in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0.) Status: Capabilities of SCT-specific TTC extension 4) The SCT-specific extension of the standard TTC system must provide the same basic capabilities as the SCT standard TTC system, namely those specified in Rule 7.6 of Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0. Justification: Notes: Rule 7.6 of Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, specifies the following two capabilities: * *Phase or timing adjustment of BC, L1A, BCR, ECR, and synchronous commands at the module or channel level as appropriate. * *Capability to send commands without introducing significant deadtime. Status: TTC timing 4) The off-detector electronics must provide all necessary timing adjustment of ATLAS TTC signals beyond the input of the SCT TTC subsystem or, if TTCrx ASICs are employed, beyond the TTCrx outputs. Justification: This requirement is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 7.5. Notes: The ATLAS TTC system provides timing adjustment up to the TTCrx outputs. The ATLAS TTC system allows timing adjustments within the off-detector electronics to be controlled by individually addressed asynchronous command and data-transmission facilities of the standard TTC system. However, at this time, it is not envisaged that the off-detector electronics will use these facilities. Comments: This requirement does not specify how the SCT TTC subsystem receives ATLAS TTC links. It is likely that standard ATLAS TTCrx receiver ASICs will be used; however, it is not clear whether these TTCrxs will be implemented at the level of the ROD crates or individual off-detector boards. Status: IV.B.2.b Interface with TTC TTC links 1) The SCT-specific TTC extension must receive TTC signals from the SCT standard TTC system on standard ATLAS TTC links. Justification: This requirement is implied by Rule 7.1 in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0 and Requirement IV.B.2.a.2 SCT standard TTC system. It defines an interface between the SCT off-detector electronics and the standard TTC system which conforms to the specification of the standard system. It is also implied by Rule 7.3. Notes: The standard TTC links will be provided by the T/DAQ group. The receivers for the standard TTC links (See Rule 7.3.), along with the SCT-specific TTC extension are the responsibility of the SCT group (See Rule 7.4.). The standard ATLAS TTC links are specified in document xxx. Comments: This requirement does not specify how the SCT TTC subsystem receives ATLAS TTC links. It is likely that standard ATLAS TTCrx receiver ASICs will be used; however, it is not clear whether these TTCrxs will be implemented at the level of the ROD crates or individual off-detector boards. Status: TTCrx ASICs 2) The SCT TTC subsystem is not required to use standard TTCrx ASICs. Justification: The interface to the SCT standard TTC system and its standard TTC links can be provided by receivers other than the TTCrx ASICs. Notes: The TTCrx ASICs will be designed by the T/DAQ group. If TTCrx ASICs are used in the SCT off-detector electronics, then they must be provided by the SCT group (See Rule 7.4 inTrigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0.) Status: IV.B.2.c Signals from TTC The following requirements describe the signals which must be received by the SCT TTC susbsytem. These requirements are dictated by Rule 7.2 of Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0. List of TTC signals 1) The SCT TTC subsystem must receive certain TTC signals from the ATLAS TTC system. These signals include: BC 40MHz clock of programmable phase BCR Bunch Crossing Reset L1A Level 1 Trigger Accept ECR Event Counter Reset ROD_L1ID 24-bit identification of L1A number ROD_BCID 12-bit identification of BC number various Synchronous Command Signals various Asynchronous caommands and data Justification: The requirement to receive standard TTC signals is imposed by the T/DAQ group, as specified in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0, Rule 4.2. It is also implied by Rule 7.1. The list of signals is derived from Rule 7.2, along with the assumption that all of these signals will be implemented in the off-detector electronics. Note: To receive certain signals means here that, if any of these signals are used in the off-detector electronics, then the internal signal implemented is derived from the standard TTC signal. It also means that, if any of these signals are not used in the off-detector electronics, then the presence of the signal(s) in the ATLAS TTC system will not cause undesirable response in the off-detector electronics. Comment: Further description of these signals and their action should be included in this document (although it is assumed that they are specified in another document). Status: The requirement to receive certain signals is well established and stable; however, the list of signals which must be received is not completely established. [for the following, state that signal is required, describe it, and describe what it is used for.] BC 2) BCR 3) L1A 4) ECR 5) Synchronous comand signals 6) Asynchronous commands and data 7) IV.B.2.d Flow control to LVL1 The ATLAS mechanism for control of data flow from front-end electronics into the Trigger/DAQ system is one of back pressure. In this model, when input buffer in any Readout Buffer (ROB) module fills, that ROB asserts an XOFF bit via the Readout Link to the Readout Driver (ROD) which sends it data. In response to the XOFF bit, the ROD stops the transmission of data. If any of the buffers of any ROD fill, such that the ROD is unable to accept further input data from detector-mounted electronics, then that ROD must assert a ROD_BUSY signal which is logically ORed with all other possible ROD_BUSY signals into a global ROD_BUSY signal which is sent to the Central Trigger Processor (CTP) in order to block the issuance of further L1A signals. Global ROD_BUSY signal 1) The off-detector electronics must provide output of a global ROD_BUSY signal to the Central Trigger Processor. Justification: This requirement is dictated by Rule 4.6 in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0. The global ROD_BUSY signal provides a mechanism to request that the Central Trigger Processor (CTP) not accept any further events (i.e. not issue any further L1A signals. This mechanism allows the off-detector electronics to protect against buffer overflow due to a variety of causes. Note: The global ROD_BUSY signal is a composite (i.e. maskable, logical OR) of individual ROD_BUSY signals generated within the off-detector electronics. Status: Protection against buffer overflow 2) The off-detector electronics may assert its global ROD_BUSY signal in order to protect its event data buffers from overflow. In determining the time to assert its global ROD_BUSY signal, the off-detector electronics must account for the number of events which may be accepted while the ROD_BUSY propagates to the Central Trigger Processor and the number of events which may already have been accepted by the Central Trigger Processor but not yet triggered in the front-end. Justification: Note: The rate at which a ROD can empty its event data buffers depends upon (1) the rate at which it can transmit data on its Readout Link(s) and (2) the rate at which the ROB(s) which receives its Readout Link(s) can receive data. (ROBs control the rate at which they receive data by use of the XOFF bit which they send to the RODs.) The rate at which the event data buffers in a ROD fill depends upon (1) the rate at which it receives input data from the detector-mounted electronics and (2) the rate at which it empties the buffers. The purpose of the ROD_BUSY signal is to provide a mechanism by which, during normal operating conditions, a ROD can avoid having its buffers overflow because its associated ROB is unable to receive data. The ROD_BUSY signal also provides a mechanism to avoid buffer overflow due to temporary large fluctuations in its input rate which cause the input rate to greatly exceed the output rate; however, the ROD should be designed so as to minimize the likelihood of such a condition. (An overall deadtime of about 0.1% is allowed due to this condition anywhere in the SCT. This overall allowance implies an allowance of only approximately 0.0004% for each ROD, assuming 256 RODs. In turn, the allowance for any one ROD input data link is 0.00001%, i.e. one part in ten million.) SCT detector-mounted electronics will occasionally (but presumably rarely) malfunction so as to produce sustained, abnormally high data rates. Such occurences should not cause deadtime for ATLAS as a whole, particularly since the data from a single input data stream is not invaluable to physics analysis. The system as a whole should provide some mechanism to promptly disable assertion of the ROD_BUSY signal associated with the malfunctioning input stream. In addition, large fluctuations in the normal (i.e no malfunctions) input data rate may be more frequent than anticipated, and hence cause more deadtime than allowed. In this circumstance, it should be possible to configure the RODs so as to not assert ROD_BUSY. The implications of the above discussion are that: .A ROD should assert its ROD_BUSY if it is receiving XOFF from its associated ROB, its buffers near overflow, and input data rates are normal on all its enabled input data links. .A ROD may assert its ROD_BUSY, even if it is not receiving XOFF, if its buffers near overflow due to a temporary large fluctuation in its input data rate. .A ROD should not assert its ROD_BUSY if it is not receiving XOFF and its buffers near overflow due to a malfunction of the detector-mounted electronics which causes an abnormally high (sustained) input data rate. In this case, it should discard the data on the malfunctioning input stream. .The ROD design should allow the possibility to enable assertion ROD_BUSY only when XOFF is asserted. It should also allow the possibility to disable assertion of ROD_BUSY even when XOFF is asserted. Status: Excessive deadtime due to overflow protection x) The off-detector electronics should not assert its global ROD_BUSY signal such that it causes excessive deadtime for ATLAS as a whole during normal operating conditions. Excessive deadtime is more than 0.1% deadtime resulting from the SCT. Normal operating conditions are conditions during physics data taking for which the L1A rate is less than 100kHz and there are no large scale malfunctions of the SCT or of other components of ATLAS. Justification: [Localized malfunctions do not so severely compromise ATLAS performance that they should be allowed to cause excessive deadtime.] [0.1% is chosen in order that several such contributions are below 1% when combined.] Note: Normal operating conditions include running at maximum LHC luminosity. Different restrictions on the allowed amount of deadtime apply when there are large scale malfunctions of the SCT or of other ATLAS components. Large scale malfunctions of the SCT are ones in which a significant localized region (greater than 1% in a single layer over the full solid angle) or a significant fraction of the full angular coverage (5% of all layers over the full solid angle) have efficiency less than 50% or ones which result in tracking efficiency for tracks passing the LVL1 electron or muon trigger to be less than 90% in a significant localized region or a significant fraction of the full angular coverage (as defined above). Large scale malfunctions of other ATLAS components, as they apply here, are ones which cause a ROB associated with the SCT to assert its XOFF signal in a sustained or frequent fashion. Limits on loss of event data x) The global ROD_BUSY mechanism should operate so as to limit loss of event data under normal operating conditions in all Readout Drivers in which the ROD_BUSY mechanism is enabled. Local loss of event data should be limited to no more than 1% on any enabled (and properly functioning) input data stream. Overall loss of event data should be limited to no more than yyy% when summed over all enabled (and properly functioning) input data streams. Justification: Note: The ROD design, particularly its buffer sizes, must be appropriate in order to allow both this requirement and the previous requirement to be simultaneously met. Clarification: For RODs whose ROD_BUSY is not fully enabled (i.e. ROD_BUSY with XOFF is enabled and ROD_BUSY without XOFF is enabled and ROD_BUSY with Malfunction is enabled), this requirement applies to the subset of event data for which ROD_BUSY is enabled. For instance, if ROD_BUSY with Malfunction is disabled, then event data on malfunctioning input data streams is not limited. Similarly, if ROD_BUSY without XOFF is disabled, then event data lost due to the input rate of data exceeding the output rate capability of the ROD is not limited. Monitoring of frequency of global ROD_BUSY 4) The off-detector electronics must monitor the frequency of the assertion of the global ROD_BUSY. Justification: Status: Monitoring of frequency of global ROD_BUSY contributions 5) The off-detector electronics must monitor the frequency of occurrence of any condition which contributes to the assertion of the global ROD_BUSY signal. Justification: Note: Conditions which can contribute to the assertion of the global ROD_BUSY signal include all the ROD_BUSY signals of individual RODs. Status: Monitoring of fractional time of global ROD_BUSY 6) The off-detector electronics should monitor the fraction of time that the global ROD_BUSY is asserted. Justification: Status: Monitoring of fractional time of global ROD_BUSY contributions 7) The off-detector electronics should monitor the fraction of time that any condition exists that contributes to the assertion of the global ROD_BUSY signal. Justification: Status: Masking of contributions to global ROD_BUSY 8) The off-detector electronics must be configurable under program control so as to provide a mechanism to disable the inclusion of conditions which can contribute to the assertion of the global ROD_BUSY signal. Justification: Status: Reset of contributions to global ROD_BUSY 9) The off-detector electronics must provide a mechanism to reset any condition Justification: Status: ROD_BUSY signal x) Each Readout Driver must provide as an output a ROD_BUSY signal which can contribute to the global ROD_BUSY condition. Justification: Status: Disable for ROD_BUSY with XOFF x) Each Readout Driver must be configurable under program control so as to provide a mechanism to disable its assertion of its ROD_BUSY signal. Justification: Status: Disable for ROD_BUSY without XOFF x) Each Readout Driver must be configurable under program control so as to provide a mechanism to disable its assertion of its ROD_BUSY signal for all conditions when XOFF is not asserted on its output Readout Link. Justification: Status: Disable for ROD_BUSY without Malfunction x) Each Readout Driver must be configurable under program control so as to provide a mechanism to disable its assertion of its ROD_BUSY signal for buffer full conditions which are caused by a malfunction of detector-mounted electronics which causes an abnormally high (sustained) input data rate on one or more of its input data streams. Justification: Status: ROD_BUSY conditions - 1 x) Each Readout Driver should assert its ROD_BUSY signal when all of the following set of conditions are satisfied: .XOFF is asserted on its output Readout Link, .Its event data buffers are too full to receive data from another event without possibility of buffer overflow, or its data buffers are so full that data from subsequent triggers is likely to overflow the buffers, .Input data rates to the Readout Driver are normal on all its enabled data links, Assertion of ROD_BUSY is enabled. Justification: Note: It is irrelevant under these circumstances whether ROD_BUSY without XOFF is enabled or disabled. It is also irrelevant under these circumstances whether ROD_BUSY without Malfunction is enabled or disabled. Status: ROD_BUSY conditions - 2 x) Each Readout Driver should assert its ROD_BUSY signal when all of the following set of conditions are satisfied: .XOFF is NOT asserted on its output Readout Link, .Its event data buffers are too full to receive data from another event without possibility of buffer overflow, or its data buffers are so full that data from subsequent triggers is likely to overflow the buffers, .The cause of the data buffers being full or nearly full is a temporary large fluctuation in its input data rate on one or more of its input data streams, Assertion of ROD_BUSY is enabled, Assertion of ROD_BUSY without XOFF is enabled. Justification: Note: It is irrelevant under these circumstances whether ROD_BUSY without Malfunction is enabled or disabled. Status: ROD_BUSY conditions - 3 x) A Readout Driver should NOT assert its ROD_BUSY signal when all of the following set of conditions are satisfied: .XOFF is NOT asserted on its output Readout Link, .Its event data buffers are too full to receive data from another event without possibility of buffer overflow, or its data buffers are so full that data from subsequent triggers is likely to overflow the buffers, The cause of the data buffers being full or nearly full is a malfunction of the detector-mounted electronics which causes an abnormally high (sustained) input data rate, .Assertion of ROD_BUSY is enabled, Assertion of ROD_BUSY without Malfunction is disabled. Justification: Note: It is irrelevant under these circumstances whether ROD_BUSY without XOFF is enabled or disabled. Under these circumstances, input event data may be lost. The ROD will flag the loss of event data. Status: x) The off-detector electronics must assert its global ROD_BUSY signal at a time which is sufficiently early to prevent loss of event data anywhere in the SCT in less than 1% of events. Note: 1% chosen to match derandomizer losses. [move] 2) Each Readout Driver must output an individual ROD_BUSY signal. Justification: This requirement is dictated by Rule 4.6 in Trigger & DAQ Interfaces with Front-End Systems: Requirements Document, Version 1.0. Status: [move to functional requirements] 3) The off-detector electronics must implement logic to combine the individual ROD_BUSY signals from all RODs in a logical OR in order to form the global ROD_BUSY signal. [move to functional requirements] 4) Each individual ROD_BUSY signal must be maskable and resettable. [move to functional requirements] 5) Each individual ROD_BUSY signal must be monitored. [move to functional requirements] 6) Each ROD must assert its ROD_BUSY signal in order to: [which is routed to the LVL1 Trigger System in order to avoid buffer overflow. Justification: The A ROD_BUSY signal is required in order to avoid buffer overflow and in order to hold off triggers while Status: text [move to functional requirements] IV.B.3 Data Acquisition System (DAQ) IV.B.3.a Configuration [Include a requirement for an SCT DAQ LAN in the functional requirements. (or does it belong here because it is required of us by the DAQ group?] DAQ configuration 1) The data acquisition aspects of the off-detector electronics must be configurable under the control of the Data Acquisition System. Justification: Note: Data acquisition aspects refers to all aspects of the off-detector electronics which relate directly to the flow of event data through the off-detector electronics from the detector-mounted electronics to the Trigger/DAQ System. For instance, data flow through the Readout Drivers and configuration of the detector-mounted electronics are data acquisition aspects. Status: IV.B.3.b Monitoring of data flow DAQ monitoring 2) The off-detector electronics must provide facility for the Data Acquisition System to monitor the flow of event data through the off-detector electronics. Justification: Status: Data flow error reporting 3) The off-detector must provide a mechanism to report errors in the flow of data to the Data Acquisition System. Justification: Status: IV.B.4 Detector Control System (DCS) [This section is the location for any requirements which there may be upon the off-detector electronics arising from DCS. However, since power supplies and monitoring are not functions of DCS, it may be the case that there are no requirements of DCS upon the off-detector electronics. On the other hand, later requirements for standalone or partitioned operation may require that the off-detector electronics be able to adjust or read voltages (e.g. threshold voltage during threshold scans).] IV.B.5 Additional Trigger/DAQ requirements IV.B.5.a Partitioning requirements The following requirements are associated with partitioning of SCT aspects of the Trigger/DAQ System from the remainder of T/DAQ in order to operate the SCT autonomously and regarding partitioning of portions of the SCT system. SCT partition 1) The off-detector electronics must provide capabilities necessary to enable operation of the SCT as an autonomous partition of the ATLAS DAQ system. Justification: Note: Capabilities required by partitioning should be detailed in further requirements. Status: Autonomous operation of SCT 2) The off-detector electronics must provide capabilities necessary to enable operation of the SCT in complete autonomy of the central ATLAS DAQ system (e.g. without connection to ROBs). Justification: Status: IV.B.5.b Data flow control and synchronization requirements For purposes of control of data flow and synchronization of event readout, the T/DAQ system plans to uniquely identify events with two identifiers: Bunch Crossing Identifier (BCID) and L1A Identifier (L1ID). BCID is a 12-bit number which gives the bunch crossing number within an LHC turn at which the event occured. L1ID is a 24-bit number which uniquely identifies the trigger number within the latency of the entire T/DAQ system. The full length BCID and L1ID must be provided from the Readout Driver level; however, shorter BCID and L1ID may be used in the front-end electronics. The following requirements regard control of the flow of data through the off-detector electronics, including error detection and recovery procedures. ROD_BCID 1) The off-detector electronics must receive the ROD_BCID from the TTC system. Justification: Status: ROD_L1ID 2) The off-detector electronics must receive the ROD_L1ID from the TTC system. Justification: Status: Check FE_BCID 3) The off-detector electronics must check that the FE_BCID numbers of event fragments of all input data streams are the same and consistent with the lower-order bits of ROD_BCID. Justification: Status: Check FE_L1ID 4) The off-detector electronics must check that the FE_L1ID numbers of event fragments of all input data streams are the same and consistent with the lower-order bits of ROD_L1ID. Justification: Status: Report FE_BCID errors 5) If the off-detector electronics identifies an inconsistent FE_BCID in an input data stream, it must flag an error in the associated output data stream. Justification: Status: Report FE_L1ID errors 6) If the off-detector electronics identifies an inconsistent FE_L1ID in an input data stream, it must flag an error in the associated output data stream. Justification: Status: Continue data-taking following synchronization error 7) If the off-detector electronics identifies an inconsistent FE_BCID or FE_L1ID, it must not stop data taking. Justification: Status: V Functional Requirements [these are requirements arising from the system-design of the off-det elex and by good engineering practice] V.A Requirements regarding the Readout Crate V.A.1 Readout Driver V.A.2 TTC Distribution Module V.A.3 CPU module V.A.4 Crate and backplane V.B Requirements regarding the SCT TTC subsystem V.C Requirements regarding the SCT DAQ subsystem V.D Requirements regarding the SCT DCS subsystem V.E Requirements regarding the SCT-specific functions in the ROBs servicing the SCT VI Glossary [defines specialized terms used in text of requirements]