MIBPM Timing Discussion

Beams-Doc-2122-v1

Minutes of the meeting to discuss timing issues for the MI BPM Upgrade, with comments added after the project meeting the following Tuesday.

When: Friday January 27, 2006 11:00 AM
Where: Dungeon
Minutes reported by: Rob Kutschke

[Editorial comments in square brackets]

  1. Introduction
  2. Flashes triggered by the BES
  3. Flashes triggered by an MIBS Event
  4. Flashes triggered by an RRBS Event
  5. User Defined Turn by Turn
  6. Do We Need to Adjust Delays for Energy Changes?
  7. Existing Software: Engineering Delays
  8. Proposal to Extend the Software
  9. A Needed Revision in the Timing Card
  10. Where Should We Put the Seam
  11. Summary and Conclusions


Introduction

Most of the meeting was about defining language. These minutes are not in the order that we discussed things but, instead, are group by topic.

We looked at an example of an acquisition spec and tried to understand what it was supposed to mean. Dave informed us that beams-doc-1996 is out of date. So in the table below I have extracted some example lines from a current version of I44.
ROW DELAY COMMAND DATUM1 DATUM1 DATUM3 DATUM4
2 0.0 TBT Enable I:device 0 (turns) 0 (buckets)
3 0.0 Flash Enable BES 0 (turns) 0 (buckets)
4 1.0 Flash Enable MIBS $74 0 (turns) 0 (buckets)

For both flash and TBT the third column is a pre-trigger delay in turns and the 4th column is a bucket number or bucket offset described below.

There are three distinct sorts of flashes:

Each will be dealt with separately.


Flashes triggered by the BES

For BES flashes, the meaning of DATUM4 is defined in terms of the following picture. Imagine that booster sends us an ideal batch, exactly the requested length in exactly the right bucket. As this batch circulates around the ring, signals will be induced on each BPM and propagate through the cables and the filter boards. DATUM4=0 means that, when the propagating beam signal reaches the front panel of the Echotek board, the first Echotek sync will also reach the front panel. This definition holds at every BPM around the ring.

The software includes a programmable set of engineering delays that can be adjusted so that when DATUM4=0 the system behaves as described above. If DATUM4=1, then the Echotek sync will be delayed by one bucket and the Echotek window will start at the beginning of the signal from the second bunch in the batch. And so on. The format of the engineering delays in the current software is described below. The delays can be adjusted using the I43 program.

The filters on the transition board have a rise time to 90% of peak of few a buckets. This does not change the picture significantly: presume that we can set the timing relative to the earliest detectable rise of a large pulse. This will have a sub-bucket resolution, which is more than is needed.

In the above we stated that the timing defines the front end of the Echotek window relative to the arrival time of the beam signal at the Echotek. We could have defined it as the middle of the Echotek window but chose not to. There are other MI controls that have a similar ambiguity and they use the front of the window as their reference. We chose to keep with that pattern in order to provide a uniform interface to the end users.

[ At the Tuesday meeting on January 31 this issue was raised again. We agreed that we should choose which ever choice is the more natural for the people who will actually use I44. Dave and Alberto will need to make this decision. For definiteness, the rest of document is written presuming the DATUM4=0 has the meaning described above. Should this decision change, all we need to do is add/subtract a constant from the engineering delays.]

There is no reference to the MI $AA marker in any of the above.

Both DATUM3 and DATUM4 must be positive. To start at bucket -1, you set DATUM4 to 586 ( presuming we are numbering 0...587). You probably also want to decrement the pre-trigger delay in order to preserve the seam in the turn by turn buffers. See also the discussion about the limitations of the timing card.

We discussed a failure mode. Suppose that the booster people transfer only one bunch and that it arrives one bucket earlier than intended. I gather that this does happen from time to time. If DATUM4=0, then the BPM system will not see this bunch. We decided that there are lots of tools to deal with this sort of situation and that, for purposes of this project, we can simply ignore it. In any event, one would normally set the value of DATUM4 to ensure that the bunch actually appears in the middle of the Echotek window. In this case timing at the level of +/- a few bunches is irrelevant. For more serious problems one can use safe mode. In normal operating one is dealing with long batches and the issue is also irrelevant.

The question of the seam will be discussed below.

The conclusion is that for BES injections we should use the existing "ignore the MI $AA marker" mode of the timing card. This mode is already well established; see, for example, Beams-doc-2115. Figure 6 in that document allows one to determine the correct timing for the BPM being tested.


Flashes triggered by an MIBS Event

Again DATUM3 is a pre-trigger delay denominated in turns.

The MI has a concept of "bucket number" relative to the MI $AA marker. The meaning of DATUM4 is the bucket number, at which to start the Echotek gate. Again this is start the Echotek gate, not the middle of the gate.

[ At the meeting on Tuesday January 31, we learned that the issue of start/middle of the Echotek gate is still open. See the comments in the section "Flashes Triggered by the BES". ]

The meaning of bucket 0 is described by the following picture: imagine that there is a single test bunch circulating in the machine. As this test bunch passes a reference point, ( at MI600 ?) the MI $AA marker is generated. This spatial position at the time the MI $AA is generated defines bucket 0. As the test bunch circulates around the ring, signals are generated at the pickups and propagate to the Echotek boards. Also the MI $AA marker propagates around the ring. The meaning of DATUM4=0 is that the following two signals should reach the front panel of the Echotek at the same time: the signal from the test bunch in bucket 0 and the first sync signal generated by the timing card. The engineering delays should be chosen to arrange the above timing when DATUM4=0. If DATUM4=1, then the sync should arrive one bucket after the test bunch. And so on.

The above picture obscures one minor detail. Suppose that the beam passes the reference point and the MI $AA is generated. Now consider an arbitrary BPM somewhere around the ring. There are BPM locations at which the MI $AA marker reaches the timing card after the beam signal has reached the Echotek. In this case the Echotek is triggered by the MI $AA marker from an earlier turn. I understand that most BPM locations work this way.

The MI $AA marker stays synchronized with the test bunch as the MI energy changes and as the bunch position is cogged. The one circumstance in which the beam and the MI $AA marker are not synchronized is during slip-stacking. At present [probably always ?] slip-stacking is only done during injections triggered by the BES. The behavior during slip stacking can be seen in Figure 4 of Beams-doc-2115.

After the meeting I asked Dave where bucket 0 was. He said to look at MI state 5, mixed mode slip-stacking and NUMI. Look at the timing just before extraction. Bunch 0 is "a few" buckets before the leading edge of the slip-stacked bunch. According to the lower right plot of Figure 6 in Beams-doc-2115, for MIBPM 412, bucket 0 occurs at a delay of about 400 half-buckets

MIBS transfer events are encoded on the MIBS clock at a fixed delay from the preceding MIBS $AA event. This delay (currently) has the same value for all MIBS transfer events. [ A word of clarification from Bob Webber: MIBS "events" are encoded on the beam synch clock carrier signal; the $AA is simply an event that occurs once per turn. The carrier frequency of the beam synch clock is MIRF/7, about 7MHz. ]

Recall that the timing card has two modes of operation: relative to the MI $AA marker and relative to the "trigger event", in this case some other MIBS event. Because the two are separated by a fixed offset, either mode will produce the same result here; the values of the engineering delays will differ by a fixed offset. There is, however, one difference: if we trigger off of the MIBS, we can start the measurement before the next MI $AA marker. If we trigger off of the MI $AA marker we have a "dead time" in which we cannot start a measurement. Our first thought is that it seems silly to design in a dead time that might cause us trouble some day, however unlikely that may be. This would lead us to choosing the "relative to the trigger event" option. We will postpone a final decision until after we have talked about flashes triggered by an RRBS event; there may be some economics in a common solution.

The question of the seam will be discussed below.


Flashes triggered by an RRBS Event

I don't think that we reached a final decision on this. After the January 31 project meeting we have reached a tentative decision and a work program to certify it.

Again DATUM3 is the pre-trigger delay, denominated in turns. Alberto initially said that he wants DATUM4 to mean the MI bucket number relative to the MI $AA marker; that is it should have the same meaning as it has flashes triggered by an MIBS event. The discussion for this starts exactly the same as the above discussion for flashes triggered by the MIBS. The two discussions diverge when we discuss the relative timing of the RRBS transfer event relative to the MI $AA marker. During transfers between the MI and the RR, the RF clocks of the two machines are phase locked and the intention is that all of their other timing signals also be phase locked. All RRBS transfer events follow the preceding RRBS $AA marker by a fixed delay. Again we are lead to the conclusion that the two timing board modes will produce equivalent results; the only difference would be a fixed offset in the engineering delays.

At the January 31 project meeting, Dave reported on a followup discussion with Greg Vogel. If we choose to configure the timing card to do timing relative to the RRBS transfer event, then we are vulnerable to future changes in the operation of the RR timing system. We will need to readjust our engineering delays every time they make changes. Therefore Dave recommends using the timing board in the mode that it triggers relative to the MI $AA marker.

Over the next week or two we will make a series of measurements to test if the system works properly when used this way. The following describes one concern. Imagine a series of anti-proton transfers from the Accumulator to the RR. Presumably each transfer is destined for a different spot in the bunch pattern within the RR. There is some uncertainty within our group regarding the timing gymnastics which make this happen. If we trigger off of the MI $AA, are all of these transfers correctly timed in? Or does the timing change depending where in the RR the transfer is destined? We will make the measurements to sort this out. We will also check similar issues in other MI states. The current guess is that everything will work fine if the timing card generates its triggers relative to the MI $AA marker.

We will also do the same checks when operating the timing card to do timing relative to the RRBS transfer event.

If it does turn out that timing relative to the $AA marker works, then we will use it. This will insulate us from future developments in the RR. In this case it makes sense to use the same timing card mode for transfers triggered by an MIBS event; this reduces the number of engineering delays that need to be maintained.


User Defined Turn by Turn

We did not talk about this at the meeting but I am adding it here for completeness. For a TBT acquisition specification, DATUM2 specifies that the arming event should be read from an ACNET device. The trigger event is always the MIBS $DA event. DATUM3 is a pre-trigger delay, denominated in turns. I believe that DATUM4 is intended to be an MI bucket number, relative to the MI $AA marker, in exactly the same sense as the it is for flashes triggered by an MIBS event.

So the user defined turn by turn is just another case of a measurement in the same class as "Flashes Triggered by an MIBS Event".


Do We Need to Adjust Delays for Energy Changes?

Do we need an energy dependent correction to the timing? The short answer is no. The long answer follows.

Again imagine that there is a single test bunch in the MI, at bucket 0. When this passes MI 600 the MI $AA marker is generated. The time for the MI $AA marker to propagate to some BPM crate is fixed; it does not depend on the energy of the beam. Now consider a BPM N buckets away from the $AA reference point; the test bunch will reach this BPM N RF cycles later. The duration of an RF cycle does depend on the energy of the machine.

Now consider two scenarios, one with the beam at high energy, with an RF frequency of fH, and one with the beam at low energy, with an RF frequency of fL. These frequencies have corresponding periods of TH=1/fH and TL=1/fL. In the high energy scenario let's say that, when the $AA marker reaches the timing card, the remaining propagation time of the beam signal, before it reaches the front panel of the Echotek, is T0; this does not depend on the machine energy. Given this information, one can compute the delay that one should give the timing card, nH, such that it generates the first sync just as the signal from the beam reaches the front panel of the Echotek,

nH= T0/TH = T0fH.

In the low energy scenario, when the $AA arrives at the house, the beam signal will be delayed by the differential travel time of the beam from the $AA reference point to the BPM. For a BPM N RF cycles around the ring, the differential travel time is,

N(TL-TH).
In this case one should set the BPM delay to nL buckets, where
nL = ( T0+N(TL-TH))/TL.
Notice that nH is expressed in ticks of the fH clock while nL is expressed in ticks of the fL clock. The difference between the two delay settings is,
nH - nL = T0( fH - fL ) + N ( 1 - fL/ fH ).
From various documents I have estimates of the frequencies of fH=53.10468 MHz and fL=52.8114 MHz. This gives,
nH - nL = T0( 0.29 ) + N ( 0.00552 )   ( for T0 in micro s).

I guess that a typical value of T0 will be a few micro seconds, giving a contribution of at most 1 bucket. The worst case for N is 588, which gives a contribution of about 3 buckets.

Therefore, if we ignore energy we will have timing errors of order 4 buckets when the machine is operated at the maximally wrong energy. This will normally be OK, as was discussed in the section about the possible failure mode in the section about "Flashes Triggered Relative to the BES".

At the meeting someone said something about a maximum shift of 7 buckets. I guess they know something more concrete about T0. This does not change the conclusions.


Existing Software: Engineering Delays

This section describes what is currently available in the software:

  1. Engineering delays, one set for protons, one set for anti-protons: These can be modified via I43. [Is it working yet?]
  2. I44 Delays
    In the discussion we called this "per state" timing. This is badly named. A given state might contain several flash or TBT commands. There is one of the following for each flash or TBT command in each state: These values are the values specified as DATUM3 and DATUM4 in I44. Be careful to correct for a unit mismatch: the software uses half-buckets internally, while the I44 user interface is in buckets.

The algorithm to use these numbers is:

  1. Add the two pre-trigger delays and load the result into the appropriate timing board register.
  2. Add the three delays denominated in half-buckets and load the result into the appropriate timing board register. See the discussion about the limitations of the current timing card.
  3. Copy the channel delays to the Echotek.

Notes:

  1. We decided that the per crate pre-trigger delay will always be set to zero. This will allow maximal control at the I44 level.


Proposal to Extend the Software

If there were only a single timing reference, the BES, for example, the above software structure would be sufficient. The engineering numbers allow us to time in the beam so that DATUM4=0 has the desired meaning. The operator can then select which part of the beam to look at by adjusting the I44 bucket delay.

This would also be true if we only used timing relative to the MI $AA marker. However it is not sufficient to support both modes of operation.

Suppose that the system is timed in for flashes triggered by the BES. If we use this system to look at a flash triggered by the $AA marker, what would it look like? Within any one crate all of the Echotek windows will be wrong by a constant offset. This can be corrected by adding an offset, denominated in half-buckets, to all channels in the crate. If we look at another crate, the Echotek windows will also be wrong by a constant offset. But the offset may differ from one crate to the next; it depends on details of cable routing and repeaters along the cable paths of the timing signals.

The delay structure needed to support this behavior is given below, with the new information in red:

  1. Engineering delays,
    1. Beam species delays, one set for protons, one set for anti-protons:
      • Pre-trigger delay, one per crate, (denominated in turns).
      • House delay, one per house ( half-buckets).
      • Board delay, one per board ( half-buckets).
      • Channel delay, one per channel (ticks of the 10/7 RF digitizer clock).
    2. Trigger source delay
      • Denominated in half-buckets, one per crate, per trigger source.
  2. I44 Delays

The change to the algorithm is to include the trigger source delay in the sum of the half-bucket delays.

My current guess is that we will have two possibilities for the initial system. We will either have two trigger sources, BES and MI $AA, or three, BES, MIBS event and RRBS event. In either case the acquisition specification has enough information for the front end to know what to do. The software design should anticipate that, in the future, there may be new trigger sources.


A Needed Revision in the Timing Card

In the proposed delay design, the half-bucket denominated delay to be loaded into the timing card is the sum of 4 contributions: the house delay, the board delay, the trigger source delay and the I44 delay. I believe that all must be positive. So there will be occasions when the total delay is more than 1176 half-buckets (one turn). Moreover it could happen that some BPMs in a crate have delays more than one turn while others have delays less than one turn.

I exchanged email with Bill after the meeting. In the present design for the timing card, this delay must be less than 1176. Two options have been suggested to deal with this. First, the front end software could add the delays mod 1176. However the pre-trigger delay may be adjusted only at the crate level. This will create multiple seams in the turn by turn data. So this option is a poor choice.

The second option is to modify the timing card to allow the time to first sync to exceed one turn. Subsequent syncs would come at intervals of one turn following the first. This changes the meaning of the half-bucket delay in the timing card. Previously it was the delay from the turn reference to the Echotek sync. Now it is the delay from the expiry of the pre-trigger delay to the first sync. Bill estimates that it would take about one week to design, implement and test this change.


Where Should Put the Seam?

The "seam" refers to the problem of correctly time ordering the data from different turn-by-turn buffers, from either a flash or a user defined turn-by-turn request. The figure below will be used to discuss this issue.

In this figure the large circle represents a synchrotron in which the particles circulate counter-clockwise. The two straight lines labeled I1 and I2 represent injection lines. The filled circle marks the origin of the $AA marker and four empty circles represent BPM locations, A, B, C, D.

Consider an injection from transfer line I1. We might set the trigger delays such that the correct order to inspect the points is the order in which the beam passed each BPM, with no wasted space at the start of the arrays: B[0], C[0], D[0], A[0], B[1], and so on, where the notation A[n] denotes the measurement at BPM A for turn n. An alternative definition might be that each array index corresponds to the same turn for all BPMs, where a turn starts and ends at the origin of the $AA marker. In this case the quantity A[0] will be noise and the meaningful array elements, in the order in which the beam passed them, are B[0], C[0], D[0], A[1], B[1], and so on.

I would say that, in the first example, "the seam" is at B while, in the second example, it is at A. Alternatively one might say that, in the second example, the seam is at the origin of the $AA marker. In order to know if any data is noise, one must know both where the seam is and where the injection line enters the ring.

A similar choice occurs for injections from beam line I2. If the injections from I1 and I2 are both controlled by MIBS events, then the delay structure has no more freedom; it is fully defined by the choices made in the previous paragraph. If the first timing choice were made, then B[0] contains noise and the good data would be found in: C[0], D[0], A[0], B[1], C[1] and so on. If, on the other hand, the second timing choice were made, then both A[0] and B[0] would contain noise, and the good data would be found in C[0], D[0], A[1], B[1], C[1] and so on.

There are, of course, many other options.

My understanding of the plan adopted at the meeting on Friday January 27 is that for all turn by turn measurements we will use the second choice shown here: the seam will be at the origin of the MI $AA marker. The proposed delay structure has enough freedom to implement this.

For injection and extraction flashes it is possible to check that the intended timing is correctly implemented by checking that, at each BPM, the beam appears, or disappears, at the correct turn relative to its neighbors. However the system will not rely on the seam being in the correct place for injection and extraction flashes. Instead the front end will look for the first, or last, turn with beam above threshold. The first, or last, turn positions reported by the front end will be the values found by this search, not the values from the expected turn.

[ I presume that, for injection and extraction flashes, we will have some diagnostic, maybe only offline, that reports if the beam appeared at the wrong turn at any BPM. ]

The one case in which understanding the seam is crucial is for hand triggered turn by turn measurements when beam is already in the machine. As noted above, this is just a subset of the "Triggered by an MIBS Event" case and it will have the same timing. So the timing for this can be verified using injections triggered by the MIBS. Also, if a hand triggered turn by turn measurement is synchronized with a small kick to the beam, it will be possible to verify that that the seam is in the correct place. But the whole point of this system is that users can count on the seam being in the right spot without needing to check it.


Summary and Conclusion

I think that we mostly know what we want to do. The remaining issues are:
  1. Comments on the above?
  2. Does anyone see a problem with the proposed delay structure?
  3. Implement the delay structure as proposed. This includes changes to the front end and to I43, which is will be used to set the values of the trigger source delays.
  4. Where exactly is bucket 0? How many buckets ahead of the slip-stacked bunch? Can we get a few more references so that we can check that they are self consistent?
  5. Will triggering on the MI $AA marker really work for transfers triggered by the RRBS. We need data to check this.
  6. Ask Bill to implement the revised timing card that allows delays to first sync longer than 1 turn.
  7. Decide on DATUM4 meaning the bucket at the start of the Echotek gate or the bucket at the center of the gate.