Home

VLDMaking e-log entries:

1) please always put most recent entries at the top;

2) give a N-ate in green; use the 'Heading 2' style so it shows up in the TOC.

3) once entry is made, please don't change it later;

4) use 

red ink : things to be done

black ink : important things in the analysis

blue ink : ongoing things

comments from people (not me): Use courier new

6) insert horizontal line at the end of each entry; 

7) save the entry once you are done;

8) update toc by clicking on it and selecting "Update Now"

TO DO:

re-read 7 series MGT and ultrascale MGT TX/RX  to understand better what I have glissed over , like pre-emphasis etc.

Asics Vs FPGA:

Documentation

VC709 connectivity KIT: https://www.xilinx.com/support/answers/54355.html

Phase II F/W Block Design: https://gitlab.cern.ch/atlas-tdaq-felix/documents/blob/master/Phase2_FW_BlockDesign/FELIXPhaseIIFW.pdf

Phase II Readout Requirements: https://edms.cern.ch/document/2166531/1

ITK Pixel Readout Task Force: https://cds.cern.ch/record/2650732/files/ATL-COM-ITK-2018-053.pdf?

RD53B :

readout resource usage and latency estimation: https://cds.cern.ch/record/2711795

DPR: https://www.xilinx.com/support/documentation/application_notes/xapp1214-drp-bridge.pdf

Link Integrity (also in HEP/Linkintegrity folder):

equalization: WP419 (https://www.xilinx.com/support/documentation/white_papers/wp419-7Series-XCVR-Equalization.pdf) 

eye diagram: https://www.edn.com/eye-diagram-basics-reading-and-applying-eye-diagrams/ , https://www.xilinx.com/support/answers/66517.html , ug576 Chap 4 (RX Marginal Analysis ) and Fig 4-21 (Eye scan state machine)

jitter: https://www.keysight.com/upload/cmc_upload/All/Download2.pdf; ug578

64b667 interlaken: http://interlakenalliance.com/wp-content/uploads/2019/12/Interlaken_Protocol_Definition_v1.2.pdf (see comparison with 6466b)

RD53A readout via FELIX: https://cds.cern.ch/record/2690124/files/ATL-COM-DAQ-2019-160.pdf

ASIC Vs FPGAs: 

 https://www.embedded.com/improve-fpga-communications-interface-clock-jitters-with-external-plls/ , 

 https://www.mouser.com/pdfdocs/clock-tree-101-timing-basics.pdf

 https://numato.com/blog/differences-between-fpga-and-asics/#:~:text=ASIC%20stands%20for%20Application%20Specific,implies%2C%20ASICs%20are%20application%20specific.&text=The%20difference%20in%20case%20of,a%20number%20of%20configurable%20blocks.

RD53A Readout DOC: https://cds.cern.ch/record/2690124/files/ATL-COM-DAQ-2019-160.pdf

Timing Analysis: ~/work/ArgonneNationalLab/HEPTiming_Analysis_of_Computer_Hardware.pdf , http://www.vlsi-expert.com/2011/03/static-timing-analysis-sta-basic-timing.html, https://en.wikipedia.org/wiki/Static_timing_analysis

TDAQ Snowmass Paper: https://arxiv.org/abs/2203.14894

VHDL/Verilog/Vivado

-Process statement and sequential execution: https://electronics.stackexchange.com/questions/372181/process-statements-and-sequential-execution-in-vhdl

-(Infer of) Latch: https://www.intel.com/content/www/us/en/programmable/quartushelp/13.0/mergedProjects/msgs/msgs/wvrfx_l2_vhdl_id_in_comb_process_holds_value.htm

 - Latch= signal preserved 

Talks

My talks

29.08.2018 (PixelReadout); https://indico.cern.ch/event/751991/

05.02.2019 (PIxel Readout): https://indico.cern.ch/event/792307/

18.02.2019 (RD53A Testing): https://indico.cern.ch/event/790617/contributions/3319980/

30.08.2019: (PixReadout): https://indico.cern.ch/event/838055/contributions/3517389/

26.09.2019 (ITK week) : https://indico.cern.ch/event/728934/

13.12.2019 (PixelReadout): https://indico.cern.ch/event/866357/

27.03.2020 (PixelReadout): https://indico.cern.ch/event/895153/

07.04.2020 (F/W meeting / External Gearbox) : https://indico.cern.ch/event/907704/

24.07.2020 (PixelReadout): https://indico.cern.ch/event/936207/

11.09.2020 (PixelReadout): https://indico.cern.ch/event/948391/

24.09.2020 (ITk Week): https://indico.cern.ch/event/950039/

23.11.2020 (Felix General): https://indico.cern.ch/event/977667/

01.12.2020 (AUW week ITK TDAQ): https://indico.cern.ch/event/976599/

18.12.2020 (PixelReadout / LPGBT readout finalized): https://indico.cern.ch/event/986346/

08.03.2021 (LTI/TTC interface FELIX): https://indico.cern.ch/event/1016173/

07.05.2021 (Deskew/Service Data) https://indico.cern.ch/event/1036079/

21.05.2021 (Optoboard V0 test) https://indico.cern.ch/event/1037631/

14.07.2021 (LpGBT Internal Emulato) https://indico.cern.ch/event/1058298/

30.07.2021 (Yarr optimization and Lpgbt readout) https://indico.cern.ch/event/1060464/

09.09.2021 (LpGBT readout with FELIX summary)   https://indico.cern.ch/event/920307/#b-430531-itk-tdaq 

28.09.2021 (SW API for ITk Pixel): https://indico.cern.ch/event/1080385/

16.11.2021 (Upgrade Week, given by Sasha): https://indico.cern.ch/event/1092013/

27.01.2022 (FELIX PDR): https://indico.cern.ch/event/1108368/

01.08.2022 (Felix general, testbeam): https://indico.cern.ch/event/1186927/

03.09/2022 (ItkPix Readout): https://indico.cern.ch/event/1192113/

13.09.2022 (ITkPix Readout): https://indico.cern.ch/event/1179940/sessions/452651/#20220913

13.09.2022 (YARR  SW Optimization for FELIX): https://indico.cern.ch/event/1179940/sessions/452651/#20220913

Other relevant talks

>Sasha

https://indico.cern.ch/event/779046/contributions/3241574/attachments/1765414/2866033/2018_11_28_FELIX_RD53A_Firmware.pdf

https://indico.cern.ch/event/779046/

https://indico.cern.ch/event/860295/contributions/3683599/attachments/1968405/3273876/2020_01_12_FELIX_RD53A_Firmware_Status.pdf (systest at LBL workshop)

>Outer Barrel workshop

24 Jun 2020: https://indico.cern.ch/event/914853/contributions/3914646/attachments/2062949/3463031/OuterBarrelWorkshop_Thielmann.pdf

> DCS

(Timon): https://indico.cern.ch/event/838560/

DCS-Upgrade Week Apr 2020 : https://indico.cern.ch/event/909598/

(ITk Pixel Read-out meeting, July 3rd 2020, Oyulmaz) https://indico.cern.ch/event/935232/timetable/

> Phase 2 Block Design

(Carsten): https://indico.cern.ch/event/780769/timetable/

(JJ Teoh F/W): https://indico.cern.ch/event/728933/contributions/3294735/attachments/1786558/2909133/Atlas_ITKWeekCERN_29Jan2019.pdf

https://gitlab.cern.ch/atlas-tdaq-felix/documents/tree/master/Phase2_FW_BlockDesign

(Pixel Meeting LpGBT emulator + FELIX/RD53B resource usage ): https://indico.cern.ch/event/859936/

>TDAQ: Mar 2019

https://indico.cern.ch/event/803708/

https://indico.cern.ch/event/805929/

>ITK week:

Feb 2020: https://indico.cern.ch/event/824593/

>Upgrade week

Apr 2020 (RD53, DCS, pixel event size, etc.): https://indico.cern.ch/event/907990/

>Kick-off Phase2 Mar 2019

https://indico.cern.ch/event/805384/

Developer week: https://indico.cern.ch/event/806346/

> RD53B 

https://indico.cern.ch/event/800821/

https://indico.cern.ch/event/805384/contributions/3356123/attachments/1814382/2964809/RB_20190319.pdf

https://indico.cern.ch/event/800821/attachments/1801048/2937708/RD53B_Main.pdf

(binary tree) https://indico.cern.ch/event/790036/contributions/3294354/attachments/1784609/2905005/11.7.18.TrackerWeek.pdf#search=anders%20ryd 

(plans decdoding, no decoding in F/W) https://indico.cern.ch/event/913724/

Readout Marius 

Upgrade Week May 12: https://indico.cern.ch/event/1149581/contributions/4863172/attachments/2441838/4183223/FELIX-ITkPixV1_20220511.pdf

ITK PIX readou: Jul 22 https://indico.cern.ch/event/1183537/

> RD53A readout system test

(CERN timeline) https://indico.cern.ch/event/871587/contributions/3675876/attachments/1971656/3280022/RD53A_readout_schedule.pdf

ITK Pixel test: 

1) https://indico.cern.ch/event/940885/ 

2) https://indico.cern.ch/event/943067/ 

3) Upgrade Week May 2022 https://indico.cern.ch/event/1149581/contributions/4860746/attachments/2443252/4186105/Preparations_towards_a_multi_module_DAQ_testbench_AUW12052022.pdf

https://indico.cern.ch/event/1149581/contributions/4860736/attachments/2442956/4186044/BM_20220512_SR1SystemTest.pdf

Liverpool/ SLAC setup https://indico.cern.ch/event/1019700/

>PCIe Gen 4 HW

https://indico.cern.ch/event/883614/

>YARR

(optimization readout) https://indico.cern.ch/event/884314/ (zijun)

>Netio

C. Gottardo 12 Jul 2021: https://indico.cern.ch/event/1058472/

>VLDB

Upgrade Week April 2020 (Ali): https://indico.cern.ch/event/909598/

>VL+/VLDB+/BOB

VLDB+ manual: https://espace.cern.ch/GBT-Project/VLDBplus/_layouts/15/start.aspx#/SitePages/Home.aspx, https://vldbplus.web.cern.ch/manual/introduction.html

setup VLDB+: 

https://espace.cern.ch/project-Versatile-Link-Plus/Shared%20Documents/Versatile%20Link%20PLUS%20Project%20V2.3.pdf

on the VDLB+ 7 links GBT, 4 links VTRX+ -> 3 in principle cannot be connected, however, only 1 link out of the 7 is connected

Configure LpGBT: https://indico.cern.ch/event/1026910/

> BOB

itk readout meeting VLDB+ breakout boards (Ali): https://indico.cern.ch/event/909095/

ITK Readout meeting (Ali) https://indico.cern.ch/event/897070/#2-vldb-break-out-board

check https://indico.cern.ch/event/950039/contributions/4024765/attachments/2109927/3549080/ItkWeek2020_ASkaf.pdf

https://indico.cern.ch/event/986346/

git repo: https://gitlab.cern.ch/askaf/vldbplus-fmc-to-minidp-breakout-board

>PDR Phase2 requirement

Phase2 PDR planning: https://indico.cern.ch/event/979650/

> Phase2 FELIX HW

Hongbin Lie: https://indico.cern.ch/event/985638/ 

Hao Xu (10.05.2021): https://indico.cern.ch/event/1034805/; 

Hao Xu (07.06.2021): https://indico.cern.ch/event/1046529/

10.06.2021: https://indico.cern.ch/event/1044952/

> Status DAQ during UAW Dec 2020 (Joern)

https://indico.cern.ch/event/977116/

> Pixel Chain

https://indico.cern.ch/event/977111/

> UVVM

UVM Vs UVVM: https://forums.xilinx.com/t5/Simulation-and-Verification/cheapest-and-easiest-way-to-use-uvm/td-p/1057217

>64b667b (Interlaken): 

LAr/TDAQ Phase2 (Najib) https://indico.cern.ch/event/996654/

White paper: 

>64b66b Hermes

https://indico.cern.ch/event/978579/contributions/4144261/attachments/2161487/3647054/Hermes_protocol.pdf

>SW Rod

Carlos 26.01.2021: https://indico.cern.ch/event/997213/

>GBT/lpGBT

IC/EC comunication based on HDLC protocol: https://pos.sissa.it/313/083/pdf

>ITK SW Review:

Joern 19 Feb 2021: https://indico.cern.ch/event/1008502/

>TTC

Options for communicating with FELIX for integration in March 2021: https://indico.cern.ch/event/1010862/ -> chose to go with option A, so we avoid TTC system for now

LTI/TTC during TDAQ week (Paschalis): https://indico.cern.ch/event/1008889/contributions/4275095/attachments/2215496/3750583/my_talk_LTI_v5.pdf

FELIX and CTP interface LTI/TTC: https://indico.cern.ch/event/1172448/contributions/4924119/attachments/2513762/4321614/2022.09.22.phase2-central-trigger.pdf

Internal FELIX/TTC meeting: https://indico.cern.ch/event/1021181/

> Fiber connectivity

L. Franconi 08 Jun 2021: https://indico.cern.ch/event/1042255/

FELIX+Optoboard+ITkPix Dec 9 2022 (Angira): https://indico.cern.ch/event/1229021/

GITLAB (VC709 design)

• • Git: https://gitlab.cern.ch/mtrovato/felixrd53a

    • copied the necessary files from ~mtrovato/FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX

    • for now included firmware/Project dir

    • test  firmware/scripts/FELIX_fullmode_top/do_implementation_VC709.tcl ; if it works erase Project dir

    • version prior to make aurora "quick"

  • quick aurora in /users/mtrovato/FELIXaurora4lanes/FELIX_latest_Oct152019_quickaurora_worksJan172020/  

    • the F/W is also in teh FELIX repo but the 2018.1 never worked properly for the vc709 (VLDB not lit)

JIRA

Pixel/YARR: https://its.cern.ch/jira/browse/FLXUSERS-433

TX/RX mapping pixel flavor F/W : https://its.cern.ch/jira/browse/FLXUSERS-494

24CH Pixel: https://its.cern.ch/jira/browse/FLXUSERS-477

fice 

         lpGBT v1 : https://its.cern.ch/jira/browse/FLXUSERS-442

         https://its.cern.ch/jira/browse/FLXUSERS-413, https://its.cern.ch/jira/browse/FLXUSERS-503

netio and felixcore: https://its.cern.ch/jira/browse/FLX-1112

data duplication felixcore: https://its.cern.ch/jira/browse/FLX-1263

LpGBT: https://its.cern.ch/jira/browse/FLX-1270,  https://its.cern.ch/jira/browse/FLX-1215

Aurora: https://its.cern.ch/jira/browse/FLX-942

LpGBT Uplinkdata sometimes wrong: https://its.cern.ch/jira/browse/FLX-1334

phase2 incompatibility with SW: https://its.cern.ch/jira/browse/FLX-1374, https://its.cern.ch/jira/browse/FLX-1380

Trickle: https://its.cern.ch/jira/browse/FLX-1431

IC/EC LpGBT: https://its.cern.ch/jira/browse/FLX-1470, https://its.cern.ch/jira/browse/FLX-1449

LpGBT 5.12 Gbps: https://its.cern.ch/jira/browse/FLX-1489

LTI/TTC interface in FELIX: https://its.cern.ch/jira/browse/FLX-1537, https://its.cern.ch/jira/browse/FLX-1314, https://its.cern.ch/jira/browse/FLX-1430,  https://its.cern.ch/jira/browse/FLX-2104, https://its.cern.ch/jira/browse/FLX-2108 (Versal)

LpGBT internal emulator: https://its.cern.ch/jira/browse/FLX-1566

YARR calPulse+trigger not landing in wupper in a deterministic way: https://its.cern.ch/jira/browse/FLX-1613

LPGBT debug info : https://its.cern.ch/jira/browse/FLX-1641

fix/optimization for pixel firmware flavor to be used with YARR: https://its.cern.ch/jira/browse/FLX-1649

SI5345 missing reset: https://its.cern.ch/jira/browse/FLX-1707

RD53B integration: https://its.cern.ch/jira/browse/FLX-1676

(TODO) FIFO to XPM libraries plus moving mem: https://its.cern.ch/jira/browse/FLX-1669

LpGBT Alignment false positive: https://its.cern.ch/jira/browse/FLX-1825

32b FromHost Hdr Data Format: https://its.cern.ch/jira/browse/FLX-1975

felix-star: https://its.cern.ch/jira/browse/FLX-1271

ITkPix Readout: https://its.cern.ch/jira/browse/FLX-1676, https://its.cern.ch/jira/browse/FLX-1989

testbeam: https://its.cern.ch/jira/browse/FLX-1744, https://its.cern.ch/jira/browse/FLXUSERS-567

link alignment with optoboard: https://its.cern.ch/jira/browse/FLX-2032

service data not observed in FELIX: https://its.cern.ch/jira/browse/FLX-1994

dcs and hit data with CB: https://its.cern.ch/jira/browse/FLX-1959

LPGBT VC709 : https://its.cern.ch/jira/browse/FLX-1533

VHDL coding style VSG: https://its.cern.ch/jira/browse/FLX-1587

Ghost packet from FELIX with ItkPix V1.1 and optosystem: https://its.cern.ch/jira/browse/FLX-2047

LPGBT Alignment issues FEC12: https://its.cern.ch/jira/browse/FLX-2038

Simulation via Free Modelim : https://its.cern.ch/jira/browse/FLX-2019

Simulation for 64b66b decoding: https://its.cern.ch/jira/browse/FLX-2106

July 27 2018 (FIrst test Felix<->RD53A )

RD53A/ LDO (default) configuration with the jumpers

- power from PS

  > measured: 1.4 V, 0.4 A (as seen from the PS)

  > increasing voltage doesn’t change much the drawn current

- power from PS and clk from SCC interface

  > clk chain: 1) felix to VLDB (versatile link demo board), 2) versatile to SSC interface, 3) SSC interface to RD53A

    - 1) optical connection, default felix F/W, communication 8b/10

      - elinkconfig -> set from-host, e-group 0, epath: 0 to 8b10b 4b (Fig. 1 W. Wu document)

      - flx-config set GBT_TXPOLARITY=0x1

      - leds on if correctly configured 

    - 2) via hdmi

      - WHERE ARE THE TEST POINTs for the clk? 

    - 3) via hdmi

      - WHERE ARE THE TEST POINTs for the clk? 

  > measured: 1.41 V, 0.548 A (as seen from the PS); 1.9 V, 0.592 A

    - doesn’t exactly match Fig 6 of RD3A manual or presentation from Jeglot Jimmy [1]. Ask them the exact config (ie: LDO, trigger provided?)

N.B: SSC interface is just transforming electrical signals from RD53A to optical

[1]: https://indico.lal.in2p3.fr/event/4815/contributions/15833/attachments/12883/15283/Testing_RD53_at_Lal_140318.pdf

Aug 13 2018 (F/W to Felix<->RD53A )

F/W locations

1) 1 TX and 1 RX link: ~/FELIX_latest_fromSasha_mod_1TX_1RXlink_gbt

2) 4 TX and 1 RX link: ~/FELIX_latest_fromSasha_mod

Changes:

- AN5, ..., AJ5, AH7,8 (clock) are pins for bank_113 (X1Y12-15)  ( ug476 ) below

  - those pins look correct from https://www.xilinx.com/support/documentation/boards_and_kits/virtex-7/vc709_Schematic_xtp213_rev1_0.pdf

- generated the aurora/gtwizard IP for X1Y12-15

  - ~/aurora_1_28Gbps_X1Y12to15_RXonly_2

  - ~/gtwizard_GTH_4_8_X1Y12to15_TXonly : bistream generation failed...to be investigated

in felix_gbt_fullmode_sfp_VC709.xdc

# SFP 1-4

set_property PACKAGE_PIN AN5 [get_ports {RX_N[1]}]

set_property PACKAGE_PIN AM7 [get_ports {RX_N[0]}]

set_property PACKAGE_PIN AL5 [get_ports {RX_N[2]}]

set_property PACKAGE_PIN AJ5 [get_ports {RX_N[3]}]

#MT

set_property PACKAGE_PIN AP3 [get_ports {TX_N[1]}]

set_property PACKAGE_PIN AN1 [get_ports {TX_N[0]}]

set_property PACKAGE_PIN AM3 [get_ports {TX_N[2]}]

set_property PACKAGE_PIN AL1 [get_ports {TX_N[3]}]

- gtwizard -> X1Y12; aurora -> X1Y15 : no more [1] but got [2]

  - f/w: ~/FELIX_latest_fromSasha_mod_1TX_1RXlink_gbt

- commented GTH_common module in gtwizard_support IP

-> needed signals (outclk, outrefclk) are passed from aurora IP to gtwizard_support in _wrapper_v7

[1]

CRITICAL WARNING: [Vivado 12-1411] Cannot set LOC property of ports, Instance g1.u2/GTH_inst[0].GTH_FM_TOP_INST/gtwizard_fullmode_TX_4_8_cpll_support_0_i/U0/gtwizard_fullmode_TX_4_8_cpll_init_i/U0/gtwizard_fullmode_TX_4_8_cpll_i\

/gt1_gtwizard_fullmode_TX_4_8_cpll_i/gthe2_i can not be placed in GTHE2_CHANNEL of site GTHE2_CHANNEL_X1Y12 because the bel is occupied by g1.u2/GTH_inst[0].GTH_FM_TOP_INST/aurora_64b66b_0_RX_i/inst/aurora_64b66b_0_RX_block_i/au\

rora_64b66b_0_RX_i/inst/aurora_64b66b_0_RX_wrapper_i/aurora_64b66b_0_RX_multi_gt_i/aurora_64b66b_0_RX_gt_inst_lane2/gthe2_i. This could be caused by bel constraint conflict [/users/mtrovato/FELIX_latest_fromSasha_mod/felix/firmw\

are/constraints/felix_gbt_fullmode_sfp_VC709.xdc:66]

[2] N.B: GTHE_COMMON bank 113 is X1Y3 (ug476 page 406)

Phase 1.1.2 IO and Clk Clean Up

ERROR: [Place 30-510] Unroutable Placement! A GTHE_COMMON / GTHE_CHANNEL clock component pair is not placed in a routable site pair. The GTHE_COMMON component can use the dedicated path between the GTHE_COMMON and the GTHE_CHANNEL if both are placed in the same clock region. If this sub optimal condition is acceptable for this design, you may use the CLOCK_DEDICATED_ROUTE constraint in the .xdc file to demote this message to a WARNING. However, the use of this override is highly discouraged. These examples can be used directly in the .xdc file to override this clock rule.

< set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets g1.u2/GTH_inst[0].GTH_FM_TOP_INST/gtwizard_fullmode_TX_4_8_cpll_support_0_i/U0/common0_i/gt0_qplloutclk_i] >

g1.u2/GTH_inst[0].GTH_FM_TOP_INST/gtwizard_fullmode_TX_4_8_cpll_support_0_i/U0/common0_i/gthe2_common_i (GTHE2_COMMON.QPLLOUTCLK) is provisionally placed by clockplacer on GTHE2_COMMON_X1Y2 -> wrong 

g1.u2/GTH_inst[0].GTH_FM_TOP_INST/gtwizard_fullmode_TX_4_8_cpll_support_0_i/U0/gtwizard_fullmode_TX_4_8_cpll_init_i/U0/gtwizard_fullmode_TX_4_8_cpll_i/gt0_gtwizard_fullmode_TX_4_8_cpll_i/gthe2_i (GTHE2_CHANNEL.QPLLCLK) is locked to GTHE2_CHANNEL_X1Y12

Clock Rule: rule_bufds_gthcommon_intelligent_pin

Status: PASS 

Rule Description: A BUFDS driving a GTHCommon must both be placed in the same or adjacent clock region

(top/bottom)

g1.u2/GTHREFCLK_1.ibufds_instq2_clk0 (IBUFDS_GTE2.O) is locked to IBUFDS_GTE2_X1Y6

g1.u2/GTH_inst[0].GTH_FM_TOP_INST/aurora_64b66b_0_RX_i/inst/aurora_64b66b_0_RX_block_i/gt_common_support/gthe2_common_i (GTHE2_COMMON.GTREFCLK0) is provisionally placed by clockplacer on GTHE2_COMMON_X1Y3 ->right

[3]:

VC709 evaluation board used: 

https://www.xilinx.com/support/documentation/boards_and_kits/virtex-7/vc709_Schematic_xtp213_rev1_0.pdf

-- 

TO DO:

- follow https://www.xilinx.com/Attachment/AR47672.pdf to have aurora IP 1 and 2 using the same transceiver for TX and RX respectively

-

sasha said that currently felix is getting a 240 Mhz clk, but we should assume in the future that it will get a 160 Mhz. This can be done manually by changing ttc output to 160

commented out hack in the aurora IP since they were not there when packaging the exdes 

 -> hacks in aurora IP are used for monitorning purposes (rxfsmdone, rxresetdone) -> I may be able to do without them

Notes:

- set_max_delay video: https://www.xilinx.com/video/hardware/false-path-delay-set-case-analysis.html

- follow https://www.xilinx.com/support/documentation/sw_manuals/xilinx2013_2/ug903-vivado-using-constraints.pdf, https://www.xilinx.com/support/documentation/sw_manuals/xilinx2013_2/ug906-vivado-design-analysis.pdf

- pag 48 The CPLL in the GTH transceiver has a nominal operating range between 1.6 GHz to 5.16 GHz.

page 55 QPLL GTH 8.0–13.1

Aug 16 2018 (Fullmode felix working)

-Program felix

-reboot pc 

-flx-init -T 3 -I 0

-elinkconfig:

to host: Egroup0 16 b -> 8b/10b enable

  - T2H (63b) ticked

from host: Egroup 0 Epath 0 4b -> 8b10b

  - EC (3f) ticked

clink TO_HOST/FROM_HOST remove emulator (E)

click CLOCK = TTC

  - elink config takes care of TOHOST/FROMFRONTEND_FANOUT from the W. W doc

- flx-config set GBT_TXPOLARITY=0x1

/users/aparamonov/work/FELIX_latest/felix/software/RD53A_test

fupload -D -G 0 -g 0 -p -w 4 RD53A_write_commands.txt

write_commands.txt generated from ./map_to_bistream RD53A_write_commands.txt RD53A_write_commands.txt

G = GBT link number

g = group number

p = e-path number

w = number of bits

verify from e-link config whethere w is set to 4

Aug 16 2018 (Reprogram felix: how to for fullmode )

- git clone ssh://get@getlab.cern.ch:7999/atlas-tdaq-felix/firmware.git  

- enable TXPOLARITY/RXPOLARITY in gtwizard IP and set TXPOLARITY=1 (inverted), RXPOLARITY=0 (normal)

  - to do: control the polarities via register as it is done in the top mode

- from vivado v2015.4(64-bit) 

  - source vivado_import_felix.tcl; 

  - source do_implementation_VC709.tcl

- programmed felix

- reboot

- source setup_FELIX-4.0.sh

- flx-init -T 1

  > set the clock

- flx-init 

  > set the links

Note: to power cycle felix -> reprogram and reboot

Questions:

1) why flx-init -T 1 -I 3 doesn't work right after rebooting the computer? But it works after -T 1

   - Attempted answer: according to felix manual (pag 14) -T 1 uses the Si5324 chip on the Vc709 board rather than Si5345 chip on TTCfx V3 (-T 3)

2) -is this needed? check /users/aparamonov/work/FELIX_latest/felix/software/flxcard/src/flx-init.cpp

Aug 16 2018 (PLL: how it works)

- Fundamentals of Phase-Locked Loops: http://www.analog.com/media/en/training-seminars/tutorials/MT-086.pdf

- Heavy lifting by

  - PD:

   - if frequency of +IN (refernce clk) is much higher than -IN (Fo?) than the input to VCO will be mostly positive, thus resulting in increased frequency after VCO (see VCO explanation)

     - N.B: the higher +IN frequency is the longer the input of VCO will be high (ie: 2A the deep at 0 will be shorter) -> higher -IN frequency

   - opposite applies if the frequency of +IN is smaller than -IN

   - if same frequency 2C -> input to VCO is 1) mostly 0 -> no increase in frequency? and 2) - and + are balanced

  - VCO: https://people.eecs.ku.edu/~callen58/501/Voltage-Controlled_Oscillator.pdf

   - need to understand a bit more, but the ideas is that there is a proportional dependence (can be linear) between input voltage and frequency

Sep 18 2018 (Changing register into RD53A)

• • ~/RD53A_test directory where codes/txt files to change RD53A registers are store

  • ./map_to_bitstream <register_map.txt> <write_commands.txt>

    • user friendly register_map.txt converted into commands to be sent via fupload

    • fupload -I0 -G0 -w4 -t 0 -D write_commands.txt

  • currently using using the default jumper cable setting (LDO mode) rather than the direct powering settings

• Timon (email from July 30, 2018) suggested to used direct power settings

• change jumper cables PWR_A, PWR_D to VDDX

• Drawn current increase quite a bit (0.560 A to 0.7 A). 

• Write into the register 68 0x00 (clk) or 0xFF (ground) I see waveforms changing. See below pictures (left: 0x00, right: 0xFF)

  • N.B: the Y scale is no huge (20 mV) but the change is there. Moreover the clock frequency is not that far ~1/2* 1/1.28 GHz ~ 400 MHz, as per 11.4.6 and 11.4.7 of RD53A Manual V 3.42

• N.B: I was probably using version of EPROC_OUT4_ENCRD53A.vhd w/o data_out bit inversion

• Tried again w/ bit inversion and the waveforms do not change. Checked again w/o inversion and it works

  • w/o inversion is firmware/output/FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_180916_16_20.bit 

Sep 18 2018 (Intermittent channel/lane_up?)

• F/W: ~/FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX

  • forgot to backup this mornig. Wrt to working version just added rx_data_i_fromgtxinterface_i, pre_rxdata_from_gtx_i in aurora_exdes

    • if needed remove them all the way to wrapper 

  • working bit file: firmware/output/FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_180917_17_41.bit 

  • lane/channel up in both 

    • ila (see below)

    •  and flx-config list | grep "GBT_ALIGNMENT_DONE"  

      • GBT_ALIGNMENT_DONE      0x00000000000f  RX ALIGNMENT DONE [47:0]

    • NB: QPLLLOCK=0 [1] 0x7730 [R  47:00]                         

    • sanity check: removed fiber from SFP cage in VC709 -> alignment down

Sep 25 2018 (IBERT for VC709)

• Followed: https://www.xilinx.com/support/documentation/boards_and_kits/vc709/2014_2/xtp234-vc709-ibert-c-2014-2.pdf

  • • From https://www.xilinx.com/products/boards-and-kits/dk-v7-vc709-g.html#documentation downloaded:           https://www.xilinx.com/member/forms/download/design-license.html?cid=390042&filename=rdf0232-vc709-ibert-c-2015-1.zip

    • now in felix02:~/vc709_ibert

      • upgraded for 2015.4

    • generated my ibert for bank 113

      • felix02:~/IBERT_X1Y15/ibert_7series_gth_0_example_1.28workextloopback_linerateseen1_25Gbpsduetogthregfclkdiff

        • modified top and xdc file as per example in vc709_ibert

          • main change: output USER_CLK programmable clock (page 30-31 ug887) to J31-J32 SMA port

            • this clock will be routed to via J25-J26, SMA connectors wired to GTH transceivers (pag 32 of ug887)

              • see picture below

              • TO DO: take a closer look at the schematics (xtp213) and understand that

              • another question: what are the banks at page13 of ug887 (0 to 39)? and why they don't appear in ug476? is this because they are not transceivers?

    • made sure that clk was at 156.25 Mhz via scope

    • external loopback done with optical fibers

    • IBERT was seeing 1.25 Gbps rather than 1.28 Gbps, as expected

      • f/w settings with GTREFCLK @ 160 Mhz

      • 156.25/160 * 1.28 Gbps = 1.25 Gbps

Sep 25 2018 (Si570 programmable oscillator on VC709)

• Followed: https://www.xilinx.com/support/documentation/boards_and_kits/vc709/2015_1/xtp240-vc709-si570-prog-c-2015-1.pdf

  • • Installed USB to UART drivers on windows ANL laptop

    • https://www.xilinx.com/support/documentation/boards_and_kits/install/ug1033-cp210x-usb-uart-install.pdf

    • Installed tera term on windows ANL laptop

      • installed on the windows anl laptop

      • just googled for one

    • From https://www.xilinx.com/products/boards-and-kits/dk-v7-vc709-g.html#documentation downloaded https://www.xilinx.com/member/forms/download/design-license.html?cid=390048&filename=rdf0238-vc709-si570-prog-c-2015-1.zip

      • now on felix02:~/VC709

    • from anywhere connected to the fpga open vivado and source freq_adjust.tcl (contains bit file and everything)

    • Downloaded SiLabs Programmable Oscillator Calculator on the ANL laptop

    • https://www.silabs.com/documents/login/software/ProgOscillatorSwInstall.zip

      • already installed in the windows ANL laptop

    • followed instructions and generated registers to be inputted into VC709 via tera term and USB UART cable

      • generated first a 200 Mhz clk

        • checked in both vio and with the scope that frequency is correct

      • generated 160.32 Mhz

        • 160.32 MHZ clk since from the TTC system the clk is expected to be 40.08x4 Mhz

      • values are

        • reg7: 0x01; reg8 : 0xc2; reg9:0xce; reg10:0x3c; reg11:0xf3; reg12=0x66

        • you should be able to see changed value in VIO/Si570_USER_CLK

    • Connected loopback fiber -> IBERT sees 1.28 Gbps when USERCLK = 160 Mhz

    • Connected fiber from RD53A via michael's board interface -> lots of errors and no link

      • TO DO: investigate

        • gtwizard?

        • External jitter cleaner

• • ***REMINDER****: IF VC709 power cycled reprogram oscillator with TERA TERM because clock will go back to its default values

    • user reg values above

200 MHz (4 ns reso scope)                                                160 Mhz (1 ns reso scope)

Sep 26 2018 (PRBS7 from RD53A to other VC709 works )

• • felix : ~FelixtoRD53A/4lanes/notbonded/FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_190523_15_28.bit 

  • usual setup for clock (flx-init -T 3), elinkconfig (using local clock rather than TTC this time)

  • (extra VC709): felix02 :~/IBERT_X1Y15/ibert_7series_gth_0_example_1.28workextloopback_linerateseen1_25Gbpsduetogthregfclkdiff

    • wrt to example design from IBERT changed

      • top file, xdc

        • USER_CLK used to feed GTREFCLK as explaned in https://sites.google.com/site/tdaqatlas/#TOC-Sep-25-2018-IBERT-for-VC709-

  • Changed the clock on the receiving VC709 to 160.32 Mhz via Oscillator calculator and tera term as explained in https://sites.google.com/site/tdaqatlas/#TOC-Sep-25-2018-Si570-programmable-oscillator-on-VC709-

    • I guess .32/160 ~ .2% difference in frequency from incoming data recovered and local clocks was not accpetable

  • configure RD53A chip to generate PRBS 7-bit for all four channels

  • register 61 0x3C; registers 68 -> 0xAA; 

    • • all lanes activated and spitting PRB7

      • 3 lanes work

      • one lane doesn't work because mike's board has polarity inverted

  • eye diagram:

    • wrt to IBERT defaults noticed that

      • RXTERMMOD=Programmable yields a way better open area (covers all the vertical (ie voltage) scale)

      • RXTERM doesn't matter

    • used finisar instead of avago transceiver modules improved open are by about 20%

Oct 1 2018 (seeing Event header from RD53A)

• • F/W: ~aurora_1_28Gbps_X1Y12_workstandaloneVC709

• work for data received in standalone VC709

• when trigger from TTC is sent I see (N.B: make sure in the TTC system trigger is enabled to Random and choose trigger rate )

  • 1e040000, 02X1ZZZZ

    • which means frame (b) from fig 53 of the manual and data x71

      • expected since newtriggerunit.vhd is configured to send x71 together with Trig ID 

      • x71 is Data_02 before the decoding

  • 1e040000, 02X0ZZZZ

    • which means frame (b) from fig 53 of the manual and data x6a

      •  6a is Data_00 before the decoding

      • not expected

Event header -> wrong dataout

Event header -> right data out

Oct 1 2018 (Procedure to autoread)

• • Configure Global pulse routing: fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_register_config_reg44.txt

    • from ./map_to_bitstream registers/RD53A_default_register_config_reg44.txt commands/RD53A_write_commands_register_config_reg44.txt

      • RD53A_default_register_config_reg44.txt = 44 0x100

    • N.B: cannot be disabled unless you power cycle the chip

      • once global pulse is sent the monitoring will continue -> read sec 7 of the manual to understand the details

  • Send Pulse: fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_GlobalPulse.txt

    • commands/RD53A_write_commands_GlobalPulse.txt = 5c 5c 6a 6c

      • see table 4 of manual (chose Data_01)

• Change default register address: fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_reg101102_6168.txt

  • • from ./map_to_bitstream registers/RD53A_reg101102_6168.txt commands/RD53A_write_commands_reg101102_6168.txt

      • registers/RD53A_reg101102_6168.txt  = 101 61; 102 68

• • Reading 68 and 61 as address

  • Reading 0...001010101 and 0...00100 as (default) values

default registers

changed registers

Oct 2 2018 (Trigger as command working)

• • f/w (felix): FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX_backupOct22018_commandtriggworks

  • f/w (extra VC709): ~/aurora_1_28Gbps_X1Y12_workstandaloneVC709

  • changed SM in cmd_top.vhd to support trigger as command

  • disable trigger from TTC

  • triggering ila on 1EXXXXXX

• fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_Trigger_tmp.txt

  • RD53A_write_commands_Trigger_tmp.txt = 2D D2 

    • 2D Trigger_2 (see Table 3)

    • D2 is Data_30 (see table 4)

  • I see:

    • trig_id = 19

      • 19 is just a counter of pending triggers in the chip (sec 9.4, fig 40, 41 of the manual)

        • see also fig cmd_15triggers to see that it's a counter from 0 to 31

          • there I send RD53A_write_commands_Trigger.txt which is all 15 triggers one after the other

      • Data_30 (decoded = D2)

cmd_2d_d2

cmd_15triggers

Oct 3 2018 (Procedure to RdReg)

• • Configure Global pulse routing: same as autoread

  • Send Pulse: same as autoread

• Set Ila Trigger on D2XX XXXX

  • Send RdReg command: fupload -I0 -G0 -w4 -t 0 -D commands/readreg1and2.txt

  • from RD53A_readreg/map_to_bitstream looping only over reg = 1, 2

  • checked that if only reg 1 to be read trigger fires only for 55 or 99 (table 11 of the manual)

  • output shows: 32b (clk100) + 32b (clk101) which decoded as per Fig 3 means:

    • addr1=2

    • value1=3

    • addr2=1

    • value2=0

    • N.B: value1,2 previously set to those values with WrReg commands

addr1=515 (ie: offset pixel row number)m value =1285 (?); addr2/val2 as above

not sure why two D2

Oct 16 2018 (Aurora data received from RD53A fully working on extra VC709, fix in Aurora IP)

• • f/w (felix): FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX_backupOct22018_commandtriggworks

• f/w (extra VC709): aurora_1_28Gbps_X1Y12_workstandaloneVC709_Oct16

  • • IMPORTANT: FIX in aurora*_wrapper.v: .bit_err_chan_bond_in ( 1'b0), //MT disable CB bit error the RD53A sequence may be different 

    • aurora IP expects Channel Bonding control characters in a precise order

    • From aurora_64b66b_0_cbcc_gtx_6466.v

//---- Logic to detect bit errors ----                                                                                                                                                                         

// By architecture, 1st CB that goes in to FIFO would Be CB, and the next                                                                                                                                        

// CB is expected to be after 9 NA idles, ie, the 11th Write should be                                                                                                                                           

// CB, Also  between 2nd to 10th writes, CB character should not be                                                                                                                                              

// written in to FIFO;   

• • l1140-1170

    • If they come differently than expected it will throw an error (bit_err_chan_bond_i)

    • error fed into aurora_64b66b_0_common_logic_cbcc, which will cause some signals/resets (e.g all_start_cb_writes_i) to go to 1

      • that will cause data to be lost: see L945-962 aurora_64b66b_0_cbcc_gtx_6466

      • do_wr_en depend on final_gater*, all_start_CB* 

        • • final_gater depends on all_start_CB*

          • if all_start_CB do_wr_en=0 and data will not be written in FIFO36E1 and will be lost (see figure)

  always @(posedge USER_CLK)

   begin

       if(CBCC_FIFO_RESET_WR_CLK)

           do_wr_en         <=  `DLY    1'b0;

         else if(wait_for_wr_en_wr4 != 2'h3)

           do_wr_en         <=  `DLY    1'b0;

       else

       begin

           if(overflow_flag_c)

               do_wr_en         <=  `DLY    1'b0;

           else if(!raw_data_r[34])

               do_wr_en         <=  `DLY    1'b0;

           else

               do_wr_en         <=  `DLY    (FINAL_GATER_FOR_FIFO_DIN | (ALL_START_CB_WRITES_IN & (cb_fifo_din_detect_q && (raw_data_r_r[34]))));

       end

   end

before fix: most of the times dor_wr_en=0 -> data lost

after fix: data all there -> correctly reach the user (rx_tdata)

example on how the rx_tdata is paused when control characters are received

example on how the tlast is asserted when 1E (seperator) is received

TO DO: ANY particular request on clock compensation in the IP Core?

Oct 25 2018 (RouterRD53A - deprecated)

• • state machine to route from Aurora to Central Router A-D frames in one GBT lane, E frame in another GBT lane

    • DRAW a diagram here

    • inputs: rx_rdata, 

  • f/w (simu): ~/RouterRD53A_testbench

  • f/w (extra VC709): ~/aurora_1_28Gbps_X1Y12_userK_littleendian

  • state machine changed so that doesn't rely on header bits any longer

    • only relying on valid, valid_k, last, data[63:0]

    • not properly working in simu (issue on data_out2)

      • synchronize valid signals with data

  • default RD53A: https://gitlab.cern.ch/mtrovato/FELIXPhase2/blob/master/aurora_64b66b_0_example_X1Y13/aurora_64b66b_0_example.srcs/sources_1/new/RouterRD53A.vhd

Oct 19 2018 (k-characters visible at user level)

• f/w: ~/aurora_1_28Gbps_X1Y12_userK_littleendian/

  • enabled in IP User K (and little endian support)

  • same procedure as https://sites.google.com/site/tdaqatlas/#TOC-Oct-3-2018-Procedure-to-RdReg-

• after 55 clk rx_user_k_tvalid goes up -> data is the same as rxdata_to_fifo_i = D2f00401 0402000 (2 clks), but stripped of D2 (kchar)

• Conclusion: can drop header bit in RouterRD53A and use k_tvalid instead

zoom on trigger

Oct 22 2018 (RX-only aurora ~works for X1Y12, doesn't work for X1Y13)

• • f/w: aurora_1_28Gbps_X1Y12RXonly_userK_littleendian

    • ~works: same results as duplex aurora

      • programmed one time -> worked

      • programmed another time -> didn't work

      • issue is described here https://sites.google.com/site/tdaqatlas/#TOC-Mar-11-2019-change-in-the-Aurora-IP-_core:-switched-to-recovered-clock-to-create-user_clk-

  • f/w: aurora_1_28Gbps_X1Y12_userK_littleendian_testX1Y13, aurora_1_28Gbps_X1Y13RXonly_userK_littleendian_fromscratch

    • doesn't work: channel_up down

    • issue is described here https://sites.google.com/site/tdaqatlas/#TOC-Mar-11-2019-change-in-the-Aurora-IP-_core:-switched-to-recovered-clock-to-create-user_clk-

  • N.B: X1Y13 in duplex mode works

Oct 23 2018 (RX aurora in FELIX working)

• • f/w: ~/FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX, https://gitlab.cern.ch/mtrovato/FELIXPhase2

  • data correctly received

    • armed ila with tlast=1

    • send a pulse

    • saw data

  • rx_lane_up

  • 0x76e0 [R 47:00] GBT_RXCDR_LOCK 0x00000500000f RX CDR LOCK [47:0]

    • 0x00000400000f if fiber disconnected from the first SFP (the one closest to the wall)

  • programmed 3 times -> same result (may need to power cycle RD53A to have rx_lane_up=1)

  • TO DO: include rx_channel_up in the register if not included

Dec 4 2018 (Procedure to readout RD53A from FELIX using ila)

***Set Clocks/links***

> flx-init -T 3

> elinkconfig

  - fromHost: Enable Epath0 w/ 4b 8b10b

  - toHost: Enable 16b 8b10b Fullmode

  - Clock : choose TTC

  - From/ToHost: remove “E”

  (flx-config set GBT_TXPOLARITY=0x1 not needed any longer)

***Check comunication***

> flx-config list | grep GBT_RXCDR_LOCK should output 0x00000500000f to show that FELIX is correctly receiving data (idles) from RD53A 

***Configure Chip and Send Cal Injection***

> source /users/mtrovato/RD53A_test/CalInjection.sh

Dec 5 2018 (Procedure to readout RD53A from FELIX using ila -> N.B: wrong configuration of the CalInjections)

• • wrt to previous F/W modified (TO DO: upload to gitlab)

  • FELIX_FM_gbt_wrapper.vhd fifo_wr_en(i) <= GT_RxByteisAligned(i) and not fifo_full(i); -> to rx_tvalid and not fifo_full(i);

  • synchronized RXdata33b and rx_tvalid_out from FullModeWrapper and aurora_exdes

  • RX_DATA_gt0_33b[32] is now an indicator whether data is Kchar (eoc, soc only) or data

  • terminal1: fdaq -D RD53A_Dec5_2 -t 10;

  • terminal2 (right after): source CalInjection.sh;

Opened FLX-card 0, firmw FM-4ch-709-1812051501-GIT:rm-4.3/66 channels=4 (cmem buffersize=1073741824)

**DEBUG ON**

Before startRec: Receiver@FLX0 DEBUG: head=0, tail=0, pwrite=0f64400000

  start=0f64400000, end=0fa4400000, pread=0f64400000(0f64400000), pcurr_addr=0f64400000, 0 0

  wraps=0, recvd=0, last-recvd=0, last-av=0, fullcnt=0

After startRec: Receiver@FLX0 DEBUG: head=0, tail=0, pwrite=0f64400000

  start=0f64400000, end=0fa4400000, pread=0f64400000(0f64400000), pcurr_addr=0f64400000, 0 0

  wraps=0, recvd=0, last-recvd=0, last-av=0, fullcnt=0

**START** using DMA #0 polling

-> 1 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 0 B, file 0 B; Buffer: 0%, wraps 0

-> 2 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 0 B, file 0 B; Buffer: 0%, wraps 0

-> 3 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 0 B, file 0 B; Buffer: 0%, wraps 0

-> 4 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0  #####start receiving data here

-> 5 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0

-> 6 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0

-> 7 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0

-> 8 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0

-> 9 sec, Rates: recv   0.0 MB/s, file   0.0 MB/s; Total: recvd 4096 B, file 4096 B; Buffer: 0%, wraps 0

• fcheck -F 1 RD53A_Dec5_2-181205-170319-1.dat

  • Notice that 88 89 71 41 89 88 00 00 matches with first 64b of waves below (red backward), 

  • c8 09 00 02 88 88 00 80 matches with the second (even though c8 09 00 do not appear in the the display)

  • ...

  • last block matches?

Chunk types: BOTH="<<", FIRST="++", LAST="&&", MIDDLE="==", TIMEOUT="]]", NULL="@@",

OUTOFBAND="##" and "TEC" for chunk truncation/error/CRCerr.

==> BLOCK 0 (E=000=0-0-0 seq=0): 1016 databytes

 88 89 71 41 89 88  0  0 c8  9  0  2 88 88  0 80 88 89  0 7c 88 89  0 78 88 88  0 74 89 88  0 70 

 89 89  0 6c 77 88  1 80 89 89  1 7c 88 89  1 78 89 89  1 74 89 88  1 70 88 89  0 68 88 88  0 64 

 89 89  1 6c 89 88 10 80 89 88 10 7c 88 88 10 78 88 88 10 74 89 88 10 70 89 88 10 6c 88 8a  1 68 

 88 88 10 68 88 88 11 80 89 88 11 7c 89 88 11 78 89 88 11 74 89 88 11 70 89 88 11 6c 88 88 11 68 

 77 89  1 64 89 89 20 80 88 88 20 7c 88 89 20 78 89 88 20 74 89 89 20 70 89 8a 20 6c 88 88 20 68 

 88 89 10 64 88 88 21 80 88 89 21 7c 89 88 21 78 89 89 21 74 88 88 21 70 89 89 21 6c 88 89 21 68 

 88 89 11 64 88 88 30 80 89 89 30 7c 88 88 30 78 88 88 30 74 88 88 30 70 88 88 30 6c 89 88 30 68 

 88 88 20 64 89 88 31 80 88 88 31 7c 88 88 31 78 88 89 31 74 89 88 31 70 89 88 31 6c 89 88 31 68 

 88 88 21 64 88 88 40 80 88 88 40 7c 89 88 40 78 88 89 40 74 88 88 40 70 89 88 40 6c 88 88 40 68 

 88 88 30 64 88 88 41 80 89 88 41 7c 88 89 41 78 88 88 41 74 88 88 41 70 88 88 41 6c 89 89 41 68 

 89 88 31 64 89 88 50 80 88 88 50 7c 88 89 50 78 88 89 50 74 89 88 50 70 89 88 50 6c 88 89 50 68 

 88 88 40 64 88 88 51 80 88 88 51 7c 88 89 51 78 88 89 51 74 88 88 51 70 89 89 51 6c 88 88 51 68 

 88 88 41 64 88 89 60 80 88 88 60 7c 88 88 60 78 88 88 60 74 88 88 60 70 88 89 60 6c 8a 88 60 68 

 89 89 50 64 88 88 61 80 89 88 61 7c 89 88 61 78 88 88 61 74 88 88 61 70 89 88 61 6c 88 89 61 68 

 88 89 51 64 89 88 70 80 88 89 70 7c 89 89 70 78 88 88 70 74 88 88 70 70 88 89 70 6c 88 88 70 68 

 88 89 60 64 89 88 71 80 89 88 71 7c 89 89 71 78 88 88 71 74 88 88 71 70 8a 88 71 6c 88 88 71 68 

 88 88 61 64 89 89 80 80 88 88 80 7c 89 88 80 78 88 89 80 74 89 88 80 70 88 89 80 6c 88 89 80 68 

 89 89 70 64 89 89 81 80 89 88 81 7c 88 88 81 78 89 89 81 74 89 88 81 70 89 88 81 6c 89 88 81 68 

 89 88 71 64 88 88 90 80 89 88 90 7c 88 88 90 78 88 88 90 74 88 88 90 70 88 89 90 6c 89 89 90 68 

 88 88 80 64 88 88 91 80 88 89 91 7c 89 89 91 78 88 88 91 74 88 88 91 70 88 88 91 6c 88 88 91 68 

 89 88 81 64 88 89 a0 80 88 89 a0 7c 88 88 a0 78 89 89 a0 74 88 89 a0 70 89 88 a0 6c 89 88 a0 68 

 88 88 90 64 89 88 a1 80 88 88 a1 7c 88 89 a1 78 89 88 a1 74 88 89 a1 70 88 89 a1 6c 88 88 a1 68 

 89 88 91 64 89 88 b0 80 88 89 b0 7c 88 88 b0 78 88 88 b0 74 88 89 b0 70 89 89 b0 6c 88 88 b0 68 

 89 88 a0 64 88 88 b1 80 88 88 b1 7c 89 89 b1 78 89 88 b1 74 88 88 b1 70 88 89 b1 6c 88 89 b1 68 

 88 88 a1 64 89 89 c0 80 89 88 c0 7c 89 89 c0 78 89 88 c0 74 89 88 c0 70 88 89 c0 6c 88 88 c0 68 

 88 89 b0 64 88 89 c1 80 88 88 c1 7c 89 89 c1 78 88 89 c1 74 89 89 c1 70 89 89 c1 6c 88 88 c1 68 

 88 89 b1 64 89 88 d0 80 88 88 d0 7c 89 89 d0 78 89 88 d0 74 88 88 d0 70 88 89 d0 6c 89 88 d0 68 

 88 88 c0 64 89 88 d1 80 89 88 d1 7c 89 88 d1 78 89 88 d1 74 88 89 d1 70 89 98 d1 6c 89 89 d1 68 

 88 89 c1 64 88 88 e0 80 89 89 e0 7c 88 89 e0 78 88 88 e0 74 88 88 e0 70 8a 88 e0 6c 88 88 e0 68 

 88 88 d0 64 88 89 e1 80 89 88 e1 7c 89 88 e1 78 89 88 e1 74 89 88 e1 70 88 88 e1 6c 89 88 e1 68 

 88 89 d1 64 88 88 f0 80 88 88 f0 7c 88 88 f0 78 89 89 f0 74 89 89 f0 70 89 88 f0 6c 88 88 f0 68 

 88 89 e0 64 88 89 f1 80 89 88 f1 7c 88 88 f1 78 89 89 f1 74 89 89 f1 70 ++ (sz=1016)

• TO DO:

  • check if the change in thFMch_fifo_driver.vhd is needed to have fdaq output data, if not revert it --MT                            CRC_error := '1';    CRC_error := '0';  --temporarily disabled 

    • • as far as I understand CRC will only cause a flag to be raised in the SW

  • deploy RD53ARouter and check whether data and register readout are properly routed to the right lanes

aurora_20MHZ

aurora_20MHZ_zoombeginning

aurora_20MHZ_zoomend

aurora_40MHZ

aurora_40MHZ_zoombeginning

aurora_40MHZ_zoombeginning

thFMch_fifo_driver

thFMch_fifo_driver_issoc_trigger

thFMch_fifo_driver_iseoc_trigger

Dec 14 2018 (Calibration working and readout makes sense for core col 15)

• • Modified CalInjection.sh

    • activating only Core Column 15 (Synchronous FE according to Fig 13 of RD53A)

    • reg32 set to x8000, register33to36 to 0x0000

    • reg2 points to col15 for calinjection

  • saved output of fdaq and ila csv on felix02:~/TESTDEC14/col15only/

  • python ReadParseCSVFile.py, I get all 2x32b messages with col = 15 (bit 0 to 5 as in Figure 53) and different rows

    • N.B: for readout we need 16b=6b for col + 6b for row + 4 for core region (Fig 57)

      • no bit for pixel pair since pixel are red in quadruplet

    • N.B: for writing we need 17b=6b for col + 6b for row + 4 for core region + 1b for pixel pair (Fig 57)

      • pixel are written in pair

  • Notes: 

    • pixel matrix is 400 (col) x 192 (row) pixels

    • 1 core region is 8 cols x 8 rows

      • in Fig 57 only 32 pixels appear => is each pixel a pair of pixel? That's our guess since the minmimum element for writing or reading is 2 pixels

    • => pixel matrix is 50 core col x 24 core row

      • sync FE for istance is 16 core col x 24 core row

• To DO: why when all columns are deactivated no signal is valid? Header and trailer should be valid

Jan 4 2019 (Plans for the BUSY signal)

• • Understand how the BUSY signal in handled in the Central router for the full mode protocol

    • gbt mode has it for sure

  • Route the 4b status code of the message readout to the BUSY signal

Jan 8 2019 (Read out registers: better described procedure)

• • As per https://sites.google.com/site/tdaqatlas/#TOC-Oct-1-2018-Procedure-to-autoread- and https://sites.google.com/site/tdaqatlas/#TOC-Oct-3-2018-Procedure-to-RdReg-

• fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_register_config_reg44.txt

• fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_GlobalPulse.txt

  • a) last byte is 3 (0xB4)

    • ila triggered on rx_user_k_tvalid = 1

      • N.B: last byte is the block_k_no which is the user k block # of table 5-3 of https://www.xilinx.com/support/documentation/ip_documentation/aurora_64b66b_protocol_spec_sp011.pdf

        • so rather than copying the all BTF byte only the block # is stored -> the other half byte is zeroed

• b) fupload -I0 -G0 -w4 -t 0 -D ../RD53A_readreg/commands/readreg1.txt

  • last byte of is 1 (0x99) or 2 (0x55)

    • ila triggered on rx_user_k_tvalid = 1 and rx_user_k_tdata[3 downto 0] != 3 to avoid autoread block_k_no 

• c) fupload -I0 -G0 -w4 -t 0 -D commands/readreg1and2.txt

  • last byte of is 0 (0xD2)

    • ila triggered on and rx_user_k_tvalid = 1 rx_user_k_tdata[3 downto 0] != 3 to avoid autoread block_k_no 

a) = 0xB4

b) = 0x99

b)  = 0x55

c) = 0xD2

Jan 11 2019 (RouterRD53A deprecated)

• • No need to use RouterRD53A (https://sites.google.com/site/tdaqatlas/#TOC-Oct-25-2018-RouterRD53A---deprecated-) any longer since

    • a,b messages in Aurora coming in one bus (rx_tdata)

    • e messages in Aurora coming in another bus (rx_user_k_tdata)

Jan 12 2018 (Procedure to auto-readout registers of RD53A from FELIX)

• • Set up FELIX: https://sites.google.com/site/tdaqatlas/#TOC-Dec-4-2018-Procedure-to-readout-RD53A-from-FELIX-using-ila-

  • Set up RD53A for Autoread: https://sites.google.com/site/tdaqatlas/#TOC-Jan-8-2018-Read-out-registers:-better-described-procedure-

  • ila results (FullModeWrapper leve)

zoom 64b

zoom 33b

• • fdaq -t 1 -D aa ; fcheck -F 1  aa..

    • output =  60 d4  9 f2  3  0  0 88

      • reading bytes backward (as usual) and forming 32b words

        • f2 09 d4 60  = 1111 0010 0000 1001 1101 0100 0110 0000 

        • 88 00 00 03 = 1000 1000 0000 0000 0000 0000 0000 0011

          • from Fig53 RD53A manual v3-42

            • 4b stas = 1111 =15 = spare

            • addr1   = 0010000010 = 130

            • val1      = 0111010100011000 = 29976 

            • addr2  = 0010001000 = 136

            • val2     = 0000 0000 0000 0000 = 0

          • Because of the aurora logic, which suppresses the most significant two bytes of a k-character (e.g: 0xb2) and introduces 0x0N, where N is the k-char # in table 5-3 of https://www.xilinx.com/support/documentation/ip_documentation/aurora_64b66b_protocol_spec_sp011.pdf I see:

            • 0000: extra

              • N= 0011 = 3 , which is 3rd kchar (=0xb4) which is both registers is autoread (Table 11 of RD53A manual v3-42)

              • explanation why aurora logic suppresses/replaces is in table 2-10 of https://www.xilinx.com/support/documentation/ip_documentation/aurora_64b66b/v11_2/pg074-aurora-64b66b.pdf

                • user_k_block_no = N (checked in the F/W)

------fdaq/fcheck output

Chunk types: BOTH="<<", FIRST="++", LAST="&&", MIDDLE="==", TIMEOUT="]]", NULL="@@",

OUTOFBAND="##" and "TEC" for chunk truncation/error/CRCerr.

==> BLOCK 0 (E=000=0-0-0 seq=1): 676 databytes

 && (sz=0)

 @@

 60 d4  9 f2  3  0  0 88 << (sz=8)

 @@

 60 d4  9 f2  3  0  0 88 << (sz=8)

 @@

Jan 12 2018 (FELIX can do both Register Readout and Cal Injection)

• • F/W: https://gitlab.cern.ch/mtrovato/felixrd53a

  • .bit/.ltx files : ~mtrovato/FelixtoRD53A/*rm-4.3_66_190115_17_44*

  • change to elinkconfig wrt default (https://sites.google.com/site/tdaqatlas/#TOC-Dec-4-2018-Procedure-to-readout-RD53A-from-FELIX-using-ila-): activate also link=1 (/users/mtrovato/RD53A_test/ConfigTwoLinks.elc)

  • .dat files (fdaq -t 10) : ~mtrovato/RD53ATestJan16/

    • Jan16_testCalInjection-190116-105538-1.dat : Sent Cal injection only

    • Jan16_testRegReadout-190116-110036-1.dat : Activated Monitorning for Register Readout only

    • Jan16_testCalInjectionRegReadout-190116-112429-1.dat : Sent Cal Injection, then activated Monitoring for Register Readout

      • fcheck -F 1000 Jan16_testCalInjectionRegReadout-190116-112429-1.dat | grep BLOCK | grep "=0-“ you will see lines from the Cal injection

      • fcheck -F 1000 Jan16_testCalInjectionRegReadout-190116-112429-1.dat | grep BLOCK | grep “=1-“ you will see lines from the Register Readout

      • N.B: if you activate Register Monitoring first and do cal injection after the cal injection is not see

  • Send a global pulse every 50usec. It's up to the user (from software) to set the reg44 to F to allow for autozeroing at the nex available global pulse

    • in testbeam reg44 won't be used for any other reason (N.B: won't need calinjection.sh and i won't need to st reg44 since I do not need to inject pixels)

      • if you need to use reg44 for other reason shut off the automatic global pulse from the f/w

  • the width set the width of the global pulse to 0 to turn it off

  • global pulse is 16b , so has to be threated in f/w as a cmd lasting 2x16b

    • create an auto_zero module that behaves as cmd_top (global_pulse case of the SM), such that generates every 4x40Mhz az_pattern_16(15 downto 0) from az_pattern(31 downto 0)

Jan 23 2018 (EPROC_OUT4_ENCRD53A cannot process two long WrReg close to each other)

CMD (BCR) correctly handled

-CMD-WrReg (4x16b) followed by CMD-AutoZero (2x16b) not correctly handled: first 16b (5c5c) of AZ read right after the WrReg and not after the send command -> hence lost

-switched to First Word Fall Throug (rather than Standard FIFO) and problem fixed (stop reading as soon as fifo_rd_en goes down) : TO SHOW

Feb 7 2018 (VLDB Configuration - windows machines)

• • Successfully done from Windows machine

    • download programmer (.jar) https://gitlab.cern.ch/gbtproj/gbtxprogrammer/tree/master/releases

      • I chose programmerv2.20180116.zip

    • download java to open the .jar : https://www.java.com/en/download/

    • to program VLDB

      • open prompt

      • cd Desktop 

      • java -jar programmerv2.20180116.jar

    • load GBTX_20180524_092606_RD53A_SASHAP.txt

      • • Import Image 

        • push Write GBTX

  • Successfully done from felix02 (as root)

    • java -jar ~mtrovato/VLDBprogramming/programmerv2.20180116.jar

    • load VLDBprogramming/GBTX_20180524_092606_RD53A_SASHAP.txt

    • more info on how to allow different user to program VLDB is here https://gitlab.cern.ch/gbtproj/gbtxprogrammer

  • Troubleshoot:

    • Use dmesg or lsusb or usb-devices to see whether the dongle is being seen

      • should appear cern among vendor device

***output of usb-devices

T:  Bus=03 Lev=01 Prnt=01 Port=02 Cnt=03 Dev#= 19 Spd=1.5 MxCh= 0

D:  Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1

P:  Vendor=16c0 ProdID=05df Rev=01.00

S:  Manufacturer=cern.ch

S:  Product=usbi2c

C:  #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=500mA

I:  If#= 0 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=00 Prot=00 Driver=usbhid

• • • dongle should have leds on when .jar is opened

Feb 18 2018 (Register readout)

• • register readout results presented here: https://indico.cern.ch/event/790617/contributions/3319980/attachments/1797767/2931695/MarcoTrovato_RD53Atesting_Feb182019.pdf

  • Addresses not increasing by 64 (slide 13) all the time is due to Aurora hand-shake

  • Slide 14 still not understood

Comment on register readout  https://anl.box.com/s/ress9lw6li08n5ki50rzt1ianv15itrp

Feb 20 2019 (to do F/W not handling tlasts in a row)

addSOPEOP

when receiving 1Edata 1Edata I will have to output SOP, 1Edata, EOP, SOP, 1Edata, EOP -> output tripled wrt to input -> put a fifo not to loose data

Se also Joseph's email

-- 

Feb 22 2019 (direct powering to LDO mode for the RD53A, Timon's visit)

• • jumpers have been moved from direct powering so we are on LDO mode (https://yarr.readthedocs.io/en/latest/rd53a/#jumper-configuration-and-power-on or Fig 1 of https://yarr.readthedocs.io/en/latest/rd53a/#jumper-configuration-and-power-on) -> PS at 1.9 V (but we measure that the voltage is 1.7 V at the entrance of the chip) and current ~0.5-0.6 A so ok

jumpers in LDO mode (

N.B: we are not in shunt-mode)

• • Configuring the chip to get lock

  • usual flx-init -T 3, elinkconfig

  • source /users/lambertj/felix/setup_joseph.sh

  • felixcore -t 1 –-data-interface lo

  • on other terminal  source /users/lambertj/felix/setup_joseph.sh

  • cd /users/lambertj/Yarr/build/

  • bin/scanConsole -r configs/controller/felix.json -c configs/connectivity/example_rd53a_setup.json -s configs/scans/rd53a/std_digitalscan.json -p

  • have to look but Timon said that contains some analogo/digital settings configurations through registers that will facilitate the locking

• if it doesn't work disconnect cable and reconnect, so voltage ramps up quickly enough

Feb 26 2019 (MGT and Aurora decoupled - add comments)

• • Pag 59 of https://www.xilinx.com/support/documentation/ip_documentation/aurora_64b66b/v11_2/pg074-aurora-64b66b.pdf states: "Generation of Aurora Without GT This option is available only in UltraScale and UltraScale+ devices"

  • F/W (VC709 standalone): /users/felix/aurora4lanes/aurora1lane_woGT_backupFeb26 

  • F/W (VC709 standalone): /users/felix/aurora4lanes/aurora1lane_woGT_worksimu 

  • GTH and protocol decoupled

Feb 27 2019 (fixed addSOPEOP in FELIX F/W

• • F/W: /users/felix/aurora4lanes/aurora1lane_woGT

  • addSOPEOP.v wasn't handling properly contiguous x1E in a row (see image)

  • fixed addSOPEOP.v : proper SM handles the flow of the data

    • data always written into FiFO

    • FIFO rd_en

      • =0 for two clocks when frame b (1E+data) containing header received

      • =0 for one clock when header or frame c (x1E no data)

      • =1 elsewhere

  • replaced the addSOPEOP in FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX and compiling

bugged addSOPEOP: you can see that two SOPs appearing without and EOP

data from cal injection (YARR) and output after adding SOP=3c and EOP=dc

zoom in data from core (Aurora IP)

zoom on data_out after adding SOP and EOP

Mar 11 2019 (Understanding how FELIX talks to the host and viceversa)

• • From User Manual flx-tools part of the flx-card which allows FELIX to talk to the Host and viceversa (fupload is from user to front-end via felix and host pc)

  • DMA = Director Memory Access (https://en.wikipedia.org/wiki/Direct_memory_access)

    • CPU free to work on something else while I/O transfer in progress. I/O and processing parallelized

    • allows I/O to happen when CPU is busy and 

  • Drivers (https://en.wikipedia.org/wiki/Device_driver): software interface to the HW mounted in the computer. It's a program which allows the CPU to talk to the HW

Mar 11 2019 (change in the aurora IP/channel_init_sm)

• • F/W (FELIX needed for tx): 

    • ~mtrovato/FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX/firmware/output/FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_190310_19_34.bit

  • F/W (extra VC709): /users/felix/aurora4lanes/aurora4lanes_woGT

    • put it on git

    • duplex mode: need to switch to RX mode only to have it working on FELIX

  • Decoupled MGT and Aurora protocol as per ultrascale example

  • Aurora 4 lanes working on extra VC709

    • lane_up/channel_up (5 LEDs on)

    • data received on all lanes

      • modified Aurora IP:

        • FIX in *channel_init_sm.vhd: chan_bond_timeout_val = 20'd20000; (rather than chan_bond_timeout_val = 9'd500;

          • watchdog reset links if bonding not achieved in 20000 (previously 500 clks)

            • not exactly sure why but that may be due to CC characters not being sent very oftend by RD53A (l545 of *cbcc_gtx.vhd)

assign valid_btf_detect_c   =  ((GTX_RX_HEADER_IN == 2'b10) &&

                                    (GTX_RX_DATA_IN[31:16] == CC_CHARACTER)

                                     && GTX_RX_DATAVALID_IN);

• • • • • FIX in aurora*_wrapper.v: bit_err_chan_bond_in ( 1'b0), //MT disable CB bit error the RD53A sequence may be different

        • see https://sites.google.com/site/tdaqatlas/#TOC-Oct-16-2018-Aurora-data-received-from-RD53A-fully-working-on-extra-VC709-fix-in-Aurora-IP-

• • • • modified ~mtrovato/RD53A_test/Calinjection.sh such to activate 4 lanes 

        • fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_reg61_60.txt

        • fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_reg69_F.txt

        • fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_reg68_55.txt

TO DO: check that every lane receives header. If so, bonding may not be needed

Mar 11 2019 (Parallelism  between Aurora_woGT and FELIX and to do in order to incorporate aurora4lanes_woGT changes into FELIX)

1) aurora4lanes_woGT: /users/felix/aurora4lanes/aurora4lanesRXonly_woGT

2) default FELIX/FullMode: (/users/mtrovato/FELIX_latest_Aug162018_Mesfininstructions/

1) aurora_64b66b_0_support.vhd -> aurora_64b66b_0_multi_wrapper.v, aurora_64b66b_0_gt_common_wrapper.vhd ->  aurora_64b66b_0_gtx.vhd, GTHE2_COMMON -> GTHE2_CHANNEL, 

2) gth_fullmode_wrapper_v7.vhd -> gth_fullmode_wrapper_v7.vhd,  GTHE2_COMMON -> (IP) gtwizard_fullmode_txcpll_rxqpll_v7.vhd, -> gtwizard_fullmode_txcpll_rxqpll_v7_init.vhd, -> gtwizard_fullmode_txcpll_rxqpll_v7_multi_gt.vhd,  -> gtwizard_fullmode_txcpll_rxqpll_v7_GT.vhd,  -> GTHE2_CHANNEL x 4, 

-GTHE2_COMMON matches for 1) and 2) -> need to externalize signals in/out gthe_common in aurora up to exdes (check also if the values are the same in two designs)

 - N.B: for the parameters all values match, except QPLL_FBDIV (check what it does). 

generate transceiver wizard with 4 GTH in duplex and hook aurora on the RX and GBT on the TX -> be careful on the gtx settings that have to match what I have in auroraRXsimplex (ie: elastic buffer)

- looking at how multi_gt is instantiated (aurora_64b66b_0_support Vs gtwizard_fullmode_txcpll_rxqpll_v7_init )

-listing things that 1) has and 2) doesn't have

 - USE_BUFG 

    - because gtwizard_fullmode_txcpll_rxqpll_v7_cpll_railing not 2) -> is this due to the fact that TX is CPLL in MESFIN's

 - WRAPPER_SIM_GTRESET_SPEEDUP (do I care?)

 - gt0_gtrefclk1_in -> create in the multi_gt and fix _gtx.v (now tied to ground, while gt0_gtrefclk0_in is getting the clk, swap those as in 1))

 - gt0_rxdisperr_out, gt0_rxnotintable_out : 8b/10b stuff don't care

 -       gt0_rxbyteisaligned_out         =>      gt_rxbyteisaligned_out(0),

       gt0_rxmcommaalignen_in          =>      '0',--gt0_rxmcommaalignen_in,                                                                                                                             

       gt0_rxpcommaalignen_in          =>      '1',--gt0_rxpcommaalignen_in,  

       gt0_rxcharisk_out

same as above

  - gt0_rxoutclkfabric_out : open anyway

-listing things that 2) has and 1) doesn't have

  -loopback

 - RXBUFSTATUS_out : open anywat

- 64b/66b specifics : check when generating gt wizard with 64b66b in rx

 //-------------------- Receive Ports - RX Gearbox Ports --------------------                                                                                                                         

         .GT0_RXDATAVALID_OUT                 (pre_rxdatavalid_i),

         .GT0_RXHEADER_OUT                    (pre_rxheader_from_gtx_i),

         .GT0_RXHEADERVALID_OUT               (pre_rxheadervalid_i),

    //------------------- Receive Ports - RX Gearbox Ports  --------------------                                                                                                                         

         .GT0_RXGEARBOXSLIP_IN                (rxgearboxslip_i),

- 64b66b specifics on TX I don't care

GT0_TXSEQUENCE_IN

   GT0_TXHEADER_IN 

- keep rx and disregard of tx (or generate gtwizard with tx at 20b no protocol and rx at 32b and see if those are there) -> they may have just not been in able when generating the wizard -> do I care?

   //----------------------- Receive Ports - CDR Ports ------------------------                                                                                                                         

         .GT0_RXCDROVRDEN_IN               (gt_rxcdrovrden_in),

         .gt0_RXPMARESET_IN                (tied_to_ground_i),

         .gt0_txpmareset_in                (tied_to_ground_i),

         .gt0_txpcsreset_in                (tied_to_ground_i),

         .gt0_rxpcsreset_in                (tied_to_ground_i),

         .gt0_rxbufreset_in                (tied_to_ground_i),

         .gt0_rxpmaresetdone_out           (),

         .gt0_txprbssel_in                 (tied_to_ground_vec_i[2:0]    ),

         .gt0_rxprbssel_in                 (tied_to_ground_vec_i[2:0]     ),

         .gt0_txprbsforceerr_in            (tied_to_ground_i),

         .gt0_rxprbserr_out                (),

         .gt0_rxprbscntreset_in            (tied_to_ground_i),

         .gt0_dmonitorout_out              (),

         .gt0_txbufstatus_out              (),

    //---------------------- TX Configurable Driver Ports ----------------------                                                                                                                     

        .GT0_txpostcursor_in               (5'b00000),

        .GT0_txdiffctrl_in                 (4'b1000),

        .GT0_txmaincursor_in               (7'b0000000),

        .gt0_txprecursor_in                (5'b00000),

        //--------------- Transmit Ports - TX Polarity Control Ports ---------------                                                                                                                     

        .GT0_txpolarity_in                 (tied_to_ground_i),

        .gt0_txinhibit_in                  (tied_to_ground_i),

    //----------- Transmit Ports - TX Initialization and Reset P

- N.B:  gt0_qplloutclk_in, gt0_qplloutrefclk_in same as gt_qpllclk_quad4_in and gt_qpllrefclk_quad4_in

- In Summary the to do in the FELIX F/W is :

1) generate gtwizard IP with TX at 20b / no encoding at 4.8 gbps, RX at 32b / 64b66b at 1.28 Gbps

2) in gth_fullmode_wrapper_v7.vhd three entities :

  a) gtwizard_fullmode_txcpll_rxqpll_v7 contains both TX and RX ins/outs

 b)GTHE2_common

 b)aurora_exdes:

   - all rx ins/outs will be connected to a)

      - remember to externalize all connections from/to aurora_64b66b_0_multi_wrapper.v

      - remove GTHE2_COMMON entity from aurora as is already in gth_fullmode_wrapper_v7.vhd

      - remove GTHE2_CHANNEL entity from aurora as is already in gth_fullmode_wrapper_v7.vhd/gtwizard_fullmode_txcpll_rxqpll_v7/gtwizard_fullmode_txcpll_rxqpll_v7.vhd -> gtwizard_fullmode_txcpll_rxqpll_v7_init.vhd -> gtwizard_fullmode_txcpll_rxqpll_v7_multi_gt.vhd  -> gtwizard_fullmode_txcpll_rxqpll_v7_GT.vhd 

- can also do the above in the extra VC709 standalone F/W for quick turnround -> there I will only need to test that the RX is not broken

Mar 11 2019 (change in the Aurora IP/*_core: switched to recovered clock to create user_clk)

• • /users/felix/aurora4lanesRXonly

  • changed IP/aurora_64b66b_0_core.v : assign tx_out_clk = rxrecclk_from_gtx_lane2_out (was = raw_tx_out_clk_i[2])

    • • bug in the IP: user_clk, which drives everything (also rx logic) was driven by TXOUTCLK. Now driven by the recovered clock

      • Question: why for the duplex mode I didn't see any problem, despite of the bug? In any case, need to propagate this change to all aurora instances

      • other changes in https://sites.google.com/site/tdaqatlas/#TOC-Oct-16-2018-Aurora-data-received-from-RD53A-fully-working-on-extra-VC709-fix-in-Aurora-IP- and https://sites.google.com/site/tdaqatlas/#TOC-Mar-11-2019-change-in-the-aurora-IP-channel_init_sm-

  • /users/felix/aurora4lanesRXonly_woGT : same change as above but the change happens in *support.v

  • Procedure to achieve lane_up/channel_up:

• • • power cycle chip

    • program extra vc709

    • (send cal injection)

    • (if you power cycle chip after programming, lane_up=channel_up=0 and you need to reprogram the extra vc709)

Mar 29 2019 (power cycling the chip sends valid data thus confusing addSOPEOP SM)

• • N.B: addSOPEOP SM from idle (=0) to evtprc (=1) when first valid data is seen

  • CalInjection.sh right after rebooting felix (data received with fdaq)

• • trigger on addSOPEOP SM changing (should be at 0 if no data received) at power cycling the chip -> state stuck at evtprc

• • CalInjection.sh after powercycling doesn't work because SM starts at state=1 (evtprc) (no fdaq received)

Mar 29 2019 (first trigger lost in YARR due to chip power cycling)

• • very first trigger lost in YARR is due to the bug in https://sites.google.com/site/tdaqatlas/#TOC-Mar-29-2019-power-cycling-the-chip-sends-valid-data-thus-confusing-addSOPEOP-SM-)

    • lost trigger details in slide 7 of https://indico.cern.ch/event/805929/contributions/3358715/attachments/1813839/2965166/2019_03_17_FELIX_RD53A_Firmware_YARR.pdf

before ...02102ae4 no SOP (ie: isSOPEOP=1 happens only after it), which means that 02102ae4 will rejected by the central router

02102ae4 not received in YARR

• • fixed by resetting the SM in addSOPEOP when rx_lane_up goes down (COMPILONG: TO BE CHECKED)

  • Sasha also asked to distinguish 

    • c frames from b frames (ie: c frame don't need the lowest significant 32b to be sent out, so I could send an EOP right away)

    • to replace 24b after 1E with xFFFFFF to avoid bug in RD53A where data is !=0 (see email to roberto on Mar 12 2019 at 11:10 am)

TO DO: in porting aurora_woGT into felix start over from fixed F/W (after the erase)

Apr 16 2019 (Local clock on FELIX)

• • elinkconfig: LOCAL clock

    • clock from internal VC709 oscillators (~40 MHZ) goes to the jitter cleaner in  TTCVx mezzanine whose output (~160 MHZ) and is routed to GTREFCLK pins via sma cables

  • no modifications to the F/W

  • TO DO : look at the clocking diagram to understand the above

Apr 17 2019 (RD53A powered from Michael's interface card)

• • RD53A powered directly from Michael's interface card

    • no need to power supply anymore (checked with single lane F/W that fdaq received data)

    • Sasha made Tim fix increase a resistor to 100 ohm for decoupling the grounds?

Apr 17 2019 (F/W for testbeam single lane)

• • F/W: /users/mtrovato/FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX_testbeam/

  • generated trigger N/P from pulse generator and hooked it up to J31,32 sma outlets of VC709

    • checked in the scope: those signals are not perfectly LVDS like (ie: P has + 1/2V offset, N has -1/2 V, which means in the difference offset doesn't cancel out, and they have some spike)

  • modified f/w to trigger N/P -> trigger  via IBUFGDS, (shown)

  • to stretch trigger for 16 40 MHZ clks (a la YARR) (shown)

  • to issue one 16 clks trigger stretched signal per input trigger pulse

  • ila outputs are below

    • the spike at clk ~170 is shown

    • at clk400 is how the F/W works: input trigger about 160 clks (=4 us), stretched signal 16 clks

Apr 22 2019 (MGT RESETDONE understood)

• • RXRESETDONE  :

    • is the final output of the reset fsm which reset sequentially each components (fig 2.21 of ug476)

  • depends on recovered clock

    • depends on rxusrrdy which is defined as follows (Table 2-28): "This port is driven High from the user's application when RXUSRCLK and RXUSRCLK2 are stable. For example, if an MMCM is used to generate both RXUSRCLK and RXUSRCLK2, then the MMCM lock signal can be used here"

    • since rxusrclk source is RXOUTCLK (Tab 3 of transceiver wizard), then to have a RXUSRCLK stable is ~RXOUTCLK stable, which is only due to QPLLCLK locked (fig 2-4 of ug476)

    • This explains why if I unplug the fibers RXRESETDONE=1

    • Q: where is the recovered clock being used?

    • Tests:

      • F/W: FELIX_latest_Aug162018_Mesfininstructions, 

      • used flx-config list -> GBT_RXRESET_DONE      0x00000000000f

      • N.B: w/ FELIX_latest_Aug272018_mod_VLDBonQPLLlock_auroraprotocoltobeins I 0x7 so one transceiver is down

      • Q: why?

Apr 27 2019 (Serial and Parallel clocks in 7 series)

• • configured transceiver wizard w/ GTREFCLK=160 MHZ, line rate = 1.28 Gbps, internal=external datawidth = 32b

  • GTHE_COMMON

    • QPLL_FBDIV =64 and QPLL_FBDIV_RATIO =1 => N=64

    • QPLL_REFCLK_DIV=1                                 => M=1

    • GTHE_CHANNEL

      • RXOUT_DIV=8                                             => D=8

    • fPLLCLKOUT = 160 x 64/(1x2) = 5.12 GHZ (Eq 2-3 ug476)

      • N.B we still comply with Table 2-2 of ug476 since VCO output frequency is 5.12 x 2 = 10.24 GHZ which is within spec

    • (fLineRate = 5.12 x 2/8 = 1.28 Gbps, Eq 2-4)

  • fPLLCLKOUT is the serial clock used to derive parallel clock (ie: RX PMA CLK or RXOUTCLK since RXOUTCLKSEL="010")

  • RXOUTCLK= fPLLCLKOUT/8/4/4 = 40 MHZ (Page 210 211 ug476)

    • 8=D

    • 4 is because RX_DATA_WIDTH=32

    • 4 is because RX_INT_DATAWIDTH=1

Notes:

• • can't change SOFT_TXRST_ALL, so in order to to see whether I can reset txstartup SM I changed the soft reset to GTTX_RESET_IN

• SOFT_RESET_TX_IN                => GTTX_RESET_IN, --MT TMP SINCE I CANNOT                                                                                                                       

                                                       --CHANGE SOFT_TXRST_ALL

reset fsm worked

aurora4lanes in FELIX

****TEST performed to have downlink working (ie: VLDB on)****

Starting from FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX

• • • -1-

    • gtwizard_fullmode_txcpll_rxqpll_v7_common IP (already) changed to support asymmetric TX/RX 

    • generated example design so II can get the GTHE2_COMMON/multi_gt (ie: channel) and put in the gth_fullmode_wrapper_v7.vhd 

    • check that common settings match with what I have in aurora 

    • continue checking gtwizard_fullmode_txcpll_rxqpll from .vho against what Kai has

    • Notes/Quesions 

      • GTHE2_COMMON

        • QPLLREFCLKSEL= 001 since I’m using GTREFCLK0 but in gtwizard_fullmode_txcpll_rxqpll_v7 I use GTREFCLK1 

        • gtwizard_fullmode_txcpll_rxqpll_v7 instance (OK stated in ug476 that only when one source is give you should use 001) 

        • GT*_TX_MMCM_LOCK_IN = 1 (as Kai does in the TX_FSM_STARTUP) (solved) 

        • no elastic buffer on RX ok? I wll switch to elastic bugger below

        • QPLLPD=0 while for aurora =1 (which means that QPLL is powered down and not used) -> how do you achieve lock in aurora? In any case at the end check if we are locked 

        • MULTI_GT 

          • uSE_BUFG=0 from Kai, not present in Aurora . I will leave it for now 

          • rx_fsm_startup using the one from KAI as it has Kai’s _gt.vhd the right rxsysclksel=11 (ie:QPLL and clock from common) 

          • rx_clk_locked=1 (used as CE in the BUFG that goes from rxrecclk_from_gtx_lane2_in to rxreclk_to_fabric_i) since Kai’s gtwizard RX_STARTUP_SM doesn’t have rx_clk_locked as outputs signals as in the aurora*support_rx_startup 

          • aurora _gt.v has rxsysclksel=txsysclksel=00 which means both TX and RX with CPLL which is ok if both TX and RX are at 1.28 Gbps or TX is disabled (pag38 of ug476)

          • gt_fullmode_wrapper_v7/aurora rx_fsm_reset_done_in => and all rxfsmtesetdone of from gtwozard 4 RX startups

• • TO DO: route the other 3 RXData_gt2,3,4 to CR

  • results: LEDS off

    • -2-

  • DRPCLK                          =>      tied_to_ground_i, --it was DRP_CLK_IN 

    • BGRCALOVRD                      =>      "00000",

    • #set_property PACKAGE_PIN AP3 [get_ports {TX_N[1]}]

    • #set_property PACKAGE_PIN AN1 [get_ports {TX_N[0]}]

    • set_property PACKAGE_PIN AP3 [get_ports {TX_N[0]}]

    • set_property PACKAGE_PIN AN1 [get_ports {TX_N[1]}]

  • results: LEDS off

  • -3-

    • started from scracth (MEsfin: /users/felix/FELIXaurora4lans/FELIX_latest_Aug162018_Mesfininstructions_mod))

    • regenereated gtwizard @160 Mhz 4.8 gbos, 1.28 gbps

    • changed _v7.vhd (added clock module, removed 8b/10b signals, etc.)

    • connected txusrclk to wordclk in wrapper.vhd

  • results: LEDS off

    • -4-

    • more chanhes:

      • gt0_gttxreset_in <= GTTX_RESET_IN(0); --MT or (not GT0_QPLLLOCK_I); and for gt1,2,3

  • results: LEDS off

  • -5-

    • more chnages

      • in GTHE2_COMMON

        • QPLLREFCLKSEL                   =>      "010", --MT Apr 10 to be consistent with how IP was generated and channel "001",     

        • GTREFCLK0 => tied_to_ground_i, --GTH_RefClk MT Apr 10 to be consistent with how IP was generated and channel 

        • GTREFCLK1 => GTH_RefClk, --tied_to_ground_i MT Apr 10 to be consistent with how IP was generated and channel , 

  • results: LEDS off

    • • GBT_RXRESET_DONE 0x00000000000f RX Reset done [47:0], GBT_RXFSMRESET_DONE 0x00000000000f RX FSM Reset done [47:0] ,GBT_PLL_LOCK_QPLL_LOCK boutncing between 1 and 0

    • -6-

    • TO ADDRESS QPLL intermittent locking

      • regenerated exdes such that refclk = 160.325 = (40.08*4, where 40.08 is the right frequency from TTC clk) to see if QPLL locks

      • QPLL not locked, but QPLL is always locked with default Kai’s F/W (*Mesifininstructions) @240 Mhz -> so I don’t think that gtrefclk=160 Mhz is the problem if 240 Mhz has no problem

    • N.B: single lane working F/W FELIX_latest_Aug272018_mod_test1link_gtwizTX_auroraRX QPLL not locked —>> WARNING!!!

      • 6) constraint file change

      • Kai is using IP @ 240 

      • QPLL not locked 

    • -7-

****starting from https://twiki.cern.ch/twiki/bin/viewauth/Sandbox/TDAQATLASANL -> produced /users/felix/ /users/felix/FELIXaurora4lans/FELIX_latest_Aug272018_mod_backApr15***

• • • externalized each component of support (gtwizard IP w/ multi_gt inside, GTHE_COMMON, common_reset, useclk_source)

    • commented common_reset module

    • results: LEDS on, QPLL lock alternating between 0 and 1

    • -8-

      • Noticed that fupload -I0 -G0 -w4 -t 0 -D commands/RD53A_write_commands_ECR.txt works if TX fiber connected to first transceiver from the motherboard. So moved TX fiber from second transceiver to first

******Test performed to have downlink working

***** /users/felix/ /users/felix/FELIXaurora4lans/FELIX_latest_Aug272018_mod****

• • • -9-

    • gt0_gttxreset_in <= GTTX_RESET_IN(0); --MT Apr15 or (not GT0_QPLLLOCK_I);

      • similar for gt1,2,3

    • reconfigured the gtwizard IP to have RX 64b/66b with internal=external data width = 32b I connected in portmap

    • added/changed

      • gt0_rxgearboxslip_i => 0

      • gt0_rxdatavalid_out => open

      • gt0_rxheader_out   => open

      • gt0_rxheadervalid_out => open

      • gt0_txheader_in => “00”

    • removed  from portmap

      • gt0_drp_busy_out                =>      open,

      • gt0_rxdisperr_out               =>      gt0_rxdisperr_out,

      • gt0_rxnotintable_out            =>      gt0_rxnotintable_out,

      • gt0_rxphmonitor_out             =>      open,               --MT gt0_rxphmonitor_out,

      • gt0_rxphslipmonitor_out         =>      open,               --MT gt0_rxphslipmonitor_out,

      • gt0_rxbyteisaligned_out         =>      gt_rxbyteisaligned_out(0), --MT

      • gt0_rxmcommaalignen_in          =>      '0',                       --MT

      • gt0_rxpcommaalignen_in          =>      '1',                       --MT

      • gt0_rxcharisk_out               =>      gt0_rxcharisk_out,

• • results: VLDB off, QPLL always lock

    • -10-

  • change in the IP (_gt.v): TXGEARBOX_EN=“FALSE” (BUG in Xilinx IP for asymmetric design)

  • results: VLDB on, QPLL always lock (checked with ila)

  • -11-

  • GEARBOX_MODE=“001” (RX for aurora)

    • => VLBD off, QPLL bouncing (switching to 320 MHZ may still make sense because N=64 will be lower)

    • -12-

    • plop in aurora in 

    • results: VLBD on, QPLL bouncing but almost stable at 1 (probed 200 times , 5 % QPLL =0), GBT_RXCDR_LOCK      0x00000000000f  RX CDR LOCK [47:0]

    • Notice that aurora standalone design (e.g: /users/felix/aurora4lanes/aurora4lanesRXonly_woGT or aurora4lanes_woGT) uses CPLL for both tx and rx (TXRX are both at 1.28 Gbps so ok) because RXSYSCLKSEL=TXSYSCLKSEL=00 in GTHCHANNEL, 

      • CPLL settings: N=16, N2=N1=4, M=1 => fPLLCLKOUT=160x(4x4/1)=2560 MHZ, fLR= 2.56x2/4=1.28 MHZ per eq 2-1, 2-2 of ug476). 

      • fPLLCLKOUT is specs (Pag 48 of ug476), LR matched to what I put in

****FELIX_latest_Aug272018_mod_gtwizardauroraprotocol signals with aurora IP*****

• • • -13-

    • in gtwizard_init.vhd commented EXAMPLE_USE_CHIPSCOPE and allowed for external reset (BUG in FELIX)

• • Fixed following WARNING: [Synth 8-3848] Net rxrecclk_from_gtx_lane2_in in module/entity aurora_64b66b_0_support does not have driver. 

  • Clock previously disconnected now connected

  • results: VLBD on, QPLL stable at 1 , GBT_RXCDR_LOCK      0x00000000000f  RX CDR LOCK [47:0], gt0,1,2 3 fsm completed, RXRESETDONE for all transceiver

  • -14-

  • in support.v user_clk_i was disconnected gt0_rxusrclk_i and replaced with user_clk_i

    • //MT Apr 23_2 commented         .USER_CLK(gt0_rxusrclk_i), //MT user_clk_i),

    • .USER_CLK(user_clk_i),

  • results: VLBD on, QPLL stable at 1 , GBT_RXCDR_LOCK      0x00000000000f  RX CDR LOCK [47:0], gt0,1,2 3 fsm completed, RXRESETDONE for all transceiver, in ila rx_lane_up bouncing but channel_up=0

  • -14-

  • noticed that /users/felix/aurora4lanes/aurora4lanesRXonly_woGT : lane/channel up after flushing bit, but down if powercycle board

  • and /users/felix/aurora4lanes/aurora4lanes_woGT : lane/channel up after flushing bit, up after power cycle the board

  • Questions: 

    • TO DO: why is duplex computed the rxpolarity ("sync_rx_polarity" in the F/W) based on the rxheader (01 = polarity 2, 10 = polarity 1 or the other way around), while the simplex doesn't?

    • why is the rxpolarity different from one lane Vs the others? Looked at Mike's schematics for the interface board and there is a mistake in pag 4 of 18pc014.pdf (GTX2 lanes have polarity inverted)

  • Hard coded rx_polarity=1 in  aurora4lanes_woGT as input of the gthe_channel for the lane0 and simplex behaves the same as duplex

    • in  FELIX_latest_Aug272018_mod_gtwizardauroraprotocol connected rxpolarities to vio w/ default value 0010

    • TO DO: HAvE POLARITIES SET FROM REGISTERS

  • results: VLBD on, QPLL stable at 1 , GBT_RXCDR_LOCK      0x00000000000f  RX CDR LOCK [47:0], gt0,1,2 3 fsm completed, RXRESETDONE for all transceiver, and 

    • FELIX_latest_Aug272018_mod_gtwizardauroraprotocol: in ila rx_lane_up  and channel_up bouncing

      • if channel_up is not achieved in a given time rx_lane_up gets reset

    • aurora4lanes_woGT : rx_lane_up=channel_up stable at 1

    • -15-

    • looked in aurora4lanesRXonly_woGT and saw that the RX startup FSM gets his soft_reset from channel bonding module (the same that reset lane_up)

  • connected soft_reset of the RX startup FSM in FELIX_latest_Aug272018_mod_gtwizardauroraprotocol to channel_bonding link reset

  • results: rx_lane_up=channel_up stable at 1

    • TO DO : DOCUMENT WAVES

sync block SM: oscillates between monitor (16) and valid (32), after 6000 correct headers (01 or 10) is done (state=1) and goes back to monitor

wrong polarity in lane0 (header =1 data !78100000 = 87EFFFFF, ...) make the channel bond fail

channel bonding failing cause the reset and thus the "lossoflink"

rxpolarity as correctly calculated and used (duplex extra VC709 F?W): as you can see lane 0 has different polarity (=1) than the others (=0) and once that is used headers of all lanes are the same (=2)

rxpolarity as wrongly calculated and used (simplex extra VC709 F?W): as you can see lane 0 has same polarity =0 as others and once that is used headers of lane0 is different (=1) than header of all other lanes (=2)

***Summary***

• • • Aurora 4 lanes with MGT decoupled and removed successfully integrated into FELIX (see https://sites.google.com/site/tdaqatlas/#TOC-May-1st-2019-Aurora-4-lanes-working-in-FELIX- )

    • While debugging found bug in 

      • Xilinx GTWIZARD IP:  for asymmetric configuration with TX no protocol and RX aurora 64b/66b: TXGEARBOXM=EN

      • FELIX: 

        • in gtwizard_init.vhd commented EXAMPLE_USE_CHIPSCOPE and allowed for external reset 

      • some registers (e.g: GTTX/RXRESET from GBT_GTTX/RX_RESET) do not work as they should or at all (see above)

      • Mike's interface card has an undocumented polarity inversion

***Additional Notes***

• FELIX:

  • GTRXRESET as the register doesn’t seem to be working

    • FULL_Mode_Data_Valid goes to 1 (see in ila with FULL_Mode_Data_Valid=1 as trigger)

      • elinkconfig is set such TH_FO not in E and not locked

        • is the emulator interfering? FULL_Mode_Data_Valid can come either from FullModeWrapper or Emulator depending on the selection register_map_control.GBT_TOHOST_FANOUT.SEL in fromfrontend_fanout_selector_FM.vhd

        • if not in E and locked FULL_Mode_Data_Valid never goes to 1

        • FHFO not locked

      • running fdaq

• test FELIX with GTREfclk at 320 MHZ

  • • added new option in flx-init to generate higher frequency clock

    • goal was to have lower N in the QPLL such that could lock easily

    • had troubled in compiling the F/W with the clock_module from aurora. Had to move from 320 to 240 with clock wizard IP(FELIX_latest_Aug272018_mod_320GTHRefclk/FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_190414_20_09.bit)

      • test didn't work

  • generated a bunch of gtwizard with different protocol selection in /users/felix/FELIXaurora4lanes/gtwizard_differentflavor to compare a) among each other and b) aurora

    • diff gtwizard_auroraprotocol_verilog/gtwizard_auroraprotocol.srcs/sources_1/ip/gtwizard_0/gtwizard_0_gt.v gtwizard_fromscratch_samesettingsasauroraprotocol/gtwizard_fromscratch_optionalportsauroraprotocol.srcs/sources_1/ip/gtwizard_0/gtwizard_0_gt.v 

<             .ALIGN_MCOMMA_DET                       ("FALSE"),

---

>             .ALIGN_MCOMMA_DET                       ("TRUE"),

283c282

<             .ALIGN_PCOMMA_DET                       ("FALSE"),

---

>             .ALIGN_PCOMMA_DET                       ("TRUE"),

• • • diff gtwizard_auroraprotocol_verilog/gtwizard_auroraprotocol.srcs/sources_1/ip/gtwizard_0/gtwizard_0_gt.v ../../aurora4lanes/aurora4lanes_woGT/aurora_64b66b_0_example/aurora_64b66b_0_example.srcs/shared_logic/aurora_64b66b_0_gtx.v

      • differences understood since aurora uses CPLL and gtwizard uses QPLL

(focus on differences at RX level)

<             .RXOUT_DIV                              (8),

---

>             .RXOUT_DIV                              (4),

May 1st 2019 (Aurora 4 lanes working in FELIX downlink , extra VC709 uplink)

• • TO BE DESCRIBED

May 1st 2019 (Aurora 4 lanes working in FELIX)

• • F/W: /users/felix/FELIXaurora4lanes/FELIX_latest_Aug272018_mod_gtwizardauroraprotocol_backMay1st_working4lanes

  • HW: describe change in configuration

  • elinkconfig: same as usual, but replicated to all 4 links

    • N.B: TH_FO locked (describe bug)

  • fdaq, fcheck

  • link number shown in red

  • data appearing in the two waveforms zoom below is bolded

==> BLOCK 0 (E=000=0-0-0 seq=0): 1012 databytes

 77 f7  6 3c 80 12  0  2 7f 77  c 3c f7 77  e 3c f7 ff 12 3c 77 f7 14 3c ff f7 18 3c 77 f7 1a 3c

 f7 ff 20 3c f7 ff 22 3c ff f7 2d 3c f7 7f 28 3c 7f ff 35 3c ff f7 37 3c 7f 7f 3d 3c ff 7f 3f 3c

 7f ff 43 3c ff 77 47 3c ff 7f 4b 3c ff f7 4d 3c f7 ff 53 3c 7f ff 55 3c f7 f7 59 3c 7f f7 5b 3c

 ff f7 6e 3c 7f f7 61 3c 77 ff 77 3c 7f ff 70 3c 7f f7 79 3c ff f7 7b 3c f7 f7 8a 3c 7f ff 8e 3c

 ff 77 92 3c ff 77 94 3c 77 ee 98 3c 77 77 9a 3c 77 f7 a7 3c 7f f7 a0 3c f7 ff ad 3c 77 7f af 3c

 77 7f b1 3c f7 77 b5 3c 7f ff c4 3c f7 ff c6 3c 7f 7f ca 3c ff f7 cc 3c 77 f7 d5 3c f7 7f d7 3c

 7f 7f db 3c ff f7 dd 3c f7 ff e3 3c 77 f7 e5 3c f7 f7 eb 3c 77 7f ed 3c 77 ff f3 3c 7f ff f5 3c

 77 ff  6 3d f7 77 fb 3c ff 7f  c 3d 7f 77  e 3d 7f ff 17 3d f7 f7 10 3d 7f e7 19 3d f7 ff 1b 3d

 7f f7 21 3d ff 7f 23 3d 7f ee 29 3d 77 f7 2b 3d ff 7f 31 3d 7f ff 33 3d 77 f7 46 3d ff 7f 39 3d

 7f 7f 4c 3d f7 ff 4e 3d 77 ff 52 3d ff f7 54 3d 7f 77 5f 3d f7 ff 58 3d ff 77 65 3d f7 f7 60 3d

 f7 ff 74 3d f7 f7 76 3d 77 f7 7c 3d 7f 77 7e 3d

==> BLOCK 1 (E=040=1-0-0 seq=0): 1012 databytes

 7f f7  7 3c fe ee  0 3c ff 7f  b 3c 77 ff  d 3c ff 77 11 3c ff 7f 13 3c ff 7f 19 3c 7f 7f 1b 3c

 ff f7 2e 3c 7f ff 23 3c f7 7f 34 3c ff f7 36 3c f7 ff 3c 3c 7f 7f 3e 3c 7f ff 44 3c f7 77 46 3c

 f7 f7 4a 3c ff 77 4c 3c 77 f7 52 3c 7f 77 54 3c f7 ff 58 3c ff 7f 5a 3c ff 7f 60 3c 77 ff 62 3c

 f7 ff 6b 3c f7 7f 68 3c f7 ff 78 3c 77 ff 7c 3c f7 77 80 3c 7f ff 82 3c 7f 77 8b 3c 7f 77 8d 3c

 7f 7f 93 3c ff f7 95 3c 77 fe 99 3c f7 7f 9b 3c 77 7f ae 3c 77 ff a1 3c 77 ff b2 3c f7 7f b4 3c

 77 fe b8 3c 77 ff ba 3c 7f 7f c5 3c 77 7f c7 3c 77 f7 d4 3c 7f ff d6 3c f7 7f da 3c f7 7f dc 3c

 7f f7 e2 3c 7f f7 e4 3c f7 7f ea 3c 77 7f ec 3c ff f7 f2 3c 7f 7f f4 3c ff 7f f8 3c f7 ff fa 3c

 7f f7  7 3d f7 7f  0 3d 7f 7f  b 3d f7 7f  f 3d 77 7f 18 3d f7 f7 11 3d 7f fe 20 3d 77 77 22 3d

 f7 f7 28 3d 77 ff 2a 3d 77 7f 30 3d 7f 7f 32 3d ef ef 38 3d 77 77 3a 3d f7 7f 45 3d 7f ff 47 3d

 ff 77 4d 3d 7f f7 4f 3d 7f f7 51 3d f7 77 53 3d f7 f7 66 3d ef ee 59 3d f7 7f 6a 3d ff 77 6e 3d

 f7 ff 75 3d 7f 7f 77 3d f7 f7 7d 3d 77 ff 7f 3d

==> BLOCK 2 (E=080=2-0-0 seq=0): 1012 databytes

 f7 7f  3 3c f7 ff  5 3c ff f7 16 3c ff 77  9 3c 7f ff 1c 3c 7f ff 1e 3c ff 77 24 3c 7f ff 26 3c

 f7 ff 2a 3c ff 7f 2c 3c 7f f7 30 3c f7 7f 32 3c ff 7f 38 3c 7f ff 3a 3c ff 77 40 3c ff 7f 42 3c

 ff 77 4f 3c ff 7f 48 3c f7 ff 57 3c ff ef 50 3c 77 7f 5d 3c ff f7 5f 3c 7f f7 65 3c ff f7 67 3c

 ff f7 72 3c f7 f7 76 3c 77 ff 7d 3c 77 7f 7f 3c ff f7 83 3c f7 f7 87 3c 77 f7 96 3c e7 ff 89 3c

 7f 7f 9e 3c 77 f7 91 3c 7f ff a4 3c 77 ff a6 3c ff 77 a8 3c f7 ff ac 3c f7 f7 b7 3c 7f ff b0 3c

 7f 77 b9 3c 7f f7 bb 3c 7f ff ce 3c f7 f7 c1 3c ff 7f d0 3c f7 77 d2 3c ff f7 df 3c ff 77 d8 3c

 f7 7f e7 3c f7 7f e0 3c 7f f7 ef 3c ee ef e8 3c 7f 7f f7 3c 7f f7 f0 3c f7 7f fd 3c ff 7f ff 3c

 7f 7f  1 3d 77 ff  3 3d f7 77 14 3d 7f f7 16 3d 77 f7 1d 3d 7f f7 1f 3d 7f ff 25 3d 77 77 27 3d

 7f 7f 2d 3d ff 77 2f 3d ff 77 35 3d 7f 7f 37 3d ff 7f 3b 3d ff f7 3f 3d f7 f7 41 3d 7f ff 43 3d

 7f f7 56 3d 7f ff 49 3d 77 ff 5a 3d f7 f7 5e 3d ff 7f 62 3d ff 77 64 3d 77 ff 6f 3d ff f7 68 3d

 f7 f7 71 3d 77 f7 73 3d ff 77 79 3d ff f7 7b 3d

==> BLOCK 3 (E=0C0=3-0-0 seq=0): 1012 databytes

 77 7f  2 3c 7f f7  4 3c f7 f7  f 3c 7f 7f  8 3c 7f f7 17 3c 77 f7 10 3c 77 ff 1d 3c ff 77 1f 3c

 7f f7 25 3c f7 ff 27 3c ff f7 29 3c ff f7 2b 3c 7f ff 31 3c f7 ff 33 3c 7f 7f 39 3c ff 7f 3b 3c

 ff f7 4e 3c f7 f7 41 3c 7f 7f 56 3c f7 ff 49 3c ff 77 5c 3c f7 ff 5e 3c 7f ff 64 3c 7f ff 66 3c

 f7 ff 6a 3c 7f 7f 6c 3c 7f 7f 7e 3c 7f ff 73 3c 77 7f 84 3c 7f ff 86 3c 7f ff 8f 3c 77 7f 88 3c

 7f f7 97 3c ff f7 90 3c f7 ff 9d 3c 77 f7 9f 3c ff 77 a3 3c 7f ff a5 3c 7f 7f b6 3c ff f7 ab 3c

 f7 7f bc 3c f7 f7 be 3c 7f 77 c0 3c ff 77 c2 3c 77 7f c9 3c 77 77 c8 3c ef ee d1 3c 77 7f d3 3c

 f7 ff e6 3c 7f ff d9 3c 7f 77 ee 3c f7 f7 e1 3c 7f ff f6 3c e7 fe e9 3c f7 7f fc 3c 77 7f fe 3c

 f7 f7  2 3d ff 7f  4 3d 77 ff  8 3d 7f f7  a 3d f7 7f 13 3d f7 ff 15 3d ff 7f 24 3d 77 ff 26 3d

 7f 7f 2c 3d 7f 7f 2e 3d f7 f7 34 3d ff 7f 36 3d 7f f7 3c 3d ff 77 3e 3d f7 f7 40 3d 7f ff 44 3d

 f7 ff 48 3d ff 7f 4a 3d f7 7f 57 3d 7f 77 50 3d 77 ff 5b 3d 7f 77 5d 3d f7 ff 61 3d f7 f7 63 3d

 7f ff 70 3d 7f f7 72 3d ef ef 78 3d f7 e7 7a 3d

May 20 2019 (1E frame appearing only in one lane)

• • F/W (FELIX): /users/felix/FELIXaurora4lanes/FELIX_latest_Aug272018_mod_gtwizardauroraprotocol

  • F/W (extra VC709) : /users/felix/aurora4lanes/aurora4lanesRXonly_woGT_tlasttvaliddecouple_backMay20

• HW: 1 RD53A only

• 1E frame come only in one lane (lane may vary from calibration to calibrations)

Jun 1 2019 (Yarr notes from TDAQ workshop)

1chip : /users/mtrovato/YarrFromJoseph/Yarr

2chips: /users/mtrovato/YarrFromJoseph/Yarr_merge

----

YARR

To compile:

download the precompiled version of the felix software and source the setup script #using Joseph's for now

setupATLAS

lsetup cmake

lsetup "gcc gcc620_x86_64_slc6"

make a build directory and cd into this

cmake3 .. -DCMAKE_TOOLCHAIN_FILE=../src/cmake/linux-gcc -DENABLE_NETIO:BOOL=ON -DNETIO_DIR:PATH=/users/lambertj/tmp/felix/felix-04-00-05/x86_64-slc6-gcc62-opt/ #using josep

make -j 20

YARR LBL

#Assumes that one has the precompiled software and the felix code from git available.

#clock/elinkconfig

cd ~/software

source /home/user14/software/cmake_tdaq/bin/setup.sh x86_64-centos7-gcc62-opt

x86_64-centos7-gcc62-opt/flxcard/flx-init

x86_64-centos7-gcc62-opt/elinkconfig/elinkconfig

YARR+felixcore

source /home/user2/felix-04-00-05/x86_64-centos7-gcc62-opt/setup.sh

• felixcore -t 1 –-data-interface lo

cd ~/Yarr_merge

source /home/user2/felix-04-00-05/x86_64-centos7-gcc62-opt/setup.sh 

cd src/build

#only the first time (or to cleanup)?

cd ../

rm -r build

cmake3 .. -DCMAKE_TOOLCHAIN_FILE=../src/cmake/linux-gcc -DENABLE_NETIO:BOOL=ON -DNETIO_DIR:PATH=/home/user2/felix-04-00-05/x86_64-centos7-gcc62-opt/

scanConsole//s->run ; ScanBase.cpp//run ; LoopActionBase.cpp//execStep(); 

1) Rd53aCoreColLoop.cpp//execPart1 (I see this appearing two times why?) ->

2) Rd53aMaskLoop.cpp // execPart1,2 (does the masking for each channel) 

-> Rd53aMaskLoop.cpp//g_tx->setCmdEnable -> NetioTxCore::setCmdEnable//enableChannel(chan) -> NetioTxCore::enableChannel

#3) Rd53aTriggerLoop.cpp//execPart1 -> g_tx->setTrigEnable(0x1) -> NetioTxCore.cpp//setTrigEnable, calls prepareTrigger-> NetioTxCore::prepareTrigger

I see 40000 events per chip which means 1 (mask stage) x 25 (core loop) 16 (#trigger) x 100 (injection). FIrst two can be decided in the json

Jun 6 2019 (Yarr and F/W working for two chips

• • F/W: /users/felix/FELIXaurora4lanes/FELIX_latest_Aug272018_mod_gtwizardauroraprotocol_working4lanesnotbonded

    • FLX709_FULLMODE_RM0403_4CH_CLKSELECT_GIT_master_rm-4.3_66_190523_15_28.bit

  • Yarr version working for two chips in here: /users/mtrovato/YarrFromJoseph/YarrTimonJun5/YARR

    • now committed to https://gitlab.cern.ch/YARR/YARR/tree/devel_rd53a_felixNetio_multichip

    • downloaded on June 5 from https://gitlab.cern.ch/YARR/YARR/tree/channel_refactor

      • fixes from Timon applied especially in the NetioTx/RxCore.cpp (https://gitlab.cern.ch/YARR/YARR/commit/3d3108a14b98a4c76be7746f83c52289c5783104)

        • I think now sends triggers and configuration messages (https://gitlab.cern.ch/YARR/YARR/blob/master/src/libNetioHW/NetioTxCore.cpp#L226 and https://gitlab.cern.ch/YARR/YARR/blob/master/src/libNetioHW/NetioTxCore.cpp#L188) one channel at the time, rather than packing messages for the two chips together

      • TO DO: compare /users/mtrovato/YarrFromJoseph/YarrTimonJun5 and /users/mtrovato/YarrFromJoseph/YARRLBL to see differences

        • YARRLBL not working for two chips, but working for one

  • Instructions to run YARR

    • felixcore (terminal 1) 

      • source /users/lambertj/tmp/felix/felix-04-00-05/x86_64-slc6-gcc62-opt/setup.sh 

      • felixcore -t 1 –-data-interface lo

    • YARR (terminal 2)

      • cd YarrFromJoseph

      • source /users/mtrovato/YarrFromJoseph/setup_joseph.sh

      • cd /users/mtrovato/YarrFromJoseph/YarrTimonJun5/YARR/src/build/

      • (#if want to start from scratch)  cd ..; rm -r build; mkdir build; cd build/ ; cmake3 .. -DCMAKE_TOOLCHAIN_FILE=../src/cmake/linux-gcc -DENABLE_NETIO:BOOL=ON -DNETIO_DIR:PATH=/users/lambertj/tmp/felix/felix-04-00-05/x86_64-slc6-gcc62-opt/ ;  make -j 20

      • bin/scanConsole -r configs/controller/felix.json -c configs/connectivity/example_rd53a_setup.json -s configs/scans/rd53a/std_digitalscan.json -p

Jun 15 2019 (Naive explanation of pixel tuning)

• • Tuning means adusting some parameters of the FEs to reduce noise w/o killing signals

  • From https://yarr.readthedocs.io/en/latest/rd53a/ Tuning of the differential FE starts from VTH1_DIFF

    • from https://indico.physics.lbl.gov/indico/event/679/contributions/1852/attachments/1618/1921/2018_07_10-LBNL_friday_meeting.pdf ( slide 7) it's obvious that VTH1 increase the global threshold and a larger injection bias is needed 

      • increasing thresholds is to get rid of the noise (https://indico.cern.ch/event/653169/contributions/2659489/attachments/1493031/2321582/talkMaria.pdf_

Jun 24 2019 (clk to drive logic always through BUFG)

• • Define enable (EN) @ clk40 (40 MHZ) as follows

    • =1 for 1 clk40 period

    • =0 for 1clk40 period

  • EN is equivalent to a 20 MHZ

  • used EN as aurora_lane_initialization clk (instead of the 20 MHZ clk): nothing worked (lane_up=0 etc...)

  • fed EN into BUFG and used the output of the BUFG as aurora_lane_initialization clk: worked

    • take home message: clk to drive logic has to be free of routing delays, which can be achieved by using BUFG

July 1 2019 (aurora latency reduced to 5 clks)

• • 5 clk latency (extra VC709): /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT or  ~/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch

    • Aurora IP cleaned up, cb modules removed (to be put back)

    • almost done migrating to a single clk domain (40 MHZ)

  • FELIX latest F/W: see above /users/felix/FELIXaurora4lanes/FELIX_latest_Aug272018_mod_gtwizardauroraprotocol_working4lanesnotbonded

******

source CalInjection.sh 6 while triggered on tvalid=1

looked rxdata_to_fifo_lane2 = 3d797fff, 3d7dfff7

rx_tdata = f7ff7d3dff7f793d (from the last byte of _to_fifo to the first byte)

****

Jul 17 2019 (aurora simulation works for reduced latency F/W)

• • /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGTworksimu

  • generate 32b data (=0xaaaaaaaa), header (=01), valids (=1) in support and passing it to the aurora core

    • bypassing data from MGT

  • works: after 72us receiving 0x55555555 (=0xaaaaaaaa reverted)

Oct 18 2019 (fixed bug in the channel bonding module: valid from MGT not considered and not starting 64b word from 0x78)

• • firmware: 

    • /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_backupOct18_cbccbugnotconsideringmgtvalidfixed_enseparatedforalllanes

  • reworked channel bonding module is converting 32b data from MGT (after unscrambling) to 64b

  • once two 32b word are accumulated EN is asserted

  • in the module I was not taking into account 

    • the invalid signal from MGT

    • nd that the 64b word should always start from 0x78

000000ada data from the trasnceiver streched because of invalid

causes data from cbcc to be 000000ada 000000ada and throw an illegal_btf

now fixed

all lanes working: first valid data

lanes (rxdata_to_fifo*) show 1 clk skew (lane1 wrt thge others)

second valid data 1/2

second valid data 2/2

• • firmware (tested): /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_backupOct22

  • firmware (tested): /users/mtrovato/FELIXaurora4lanes/FELIX_latest_Oct152019_quickaurora_backupOct22

  • Corrected small bug:

 always @(posedge CLK40)

     if(RESET)

       begin

          count_1b <= `DLY 1'b0;

          first_time <= `DLY 1'b1;

       end

     else if (fifo_din_i[36]) //valid                                                                                                           

       begin

       if (first_time && fifo_din_i[31:24] == 8'h78) //start counter if first idle seen                                                         

         begin

            count_1b <= `DLY 1'b1;

            first_time <= `DLY 1'b0;

         end

       else if (!first_time) //count   -----> bug fix                                                                                                         

         begin

            if (count_1b == 1'b0)

              begin

                 count_1b <= `DLY 1'b1;

              end

            else

              begin

                 count_1b <= `DLY  1'b0;

              end

         end

       end

     else

       begin

          count_1b <= `DLY count_1b;

       end // if (first_time and fifo_din_in[31:24] = 0x"78")                                                                                   

• • Procedure:

    • Power Cycle RD53A

    • Connect fibers such that lane0 chip goes to MGT2 felix, and the polarity inverted lane goes to the right place

      • look at data from MGT to see if any polarity should be inverted (0x78... -> ok)

    • RESET RXFSM via vio (0xF, 0x0)

    • make sure no soft_err are thrown and lane_up = f in ila

    • set up triggers as in the screenshots

    • terminal 1: fdaq - t 20 -D blabla;  terminal 2: source CalInjection.sh 6

  • Results (repeated twice; fdaq outputs 

  • testOct22_quickaurora_2-191022-132455-1.dat, 

  • testOct22_quickaurora_3-191022-134545-1.dat)

After 5 clks MGT data seen correctly (rx_tdata)

-> What's the invalid garbage (valid=0) a48... at clk 999?

-> why 003c rather than 103c? Bit flip not there if I repeat the measurement

data correctly outputted by my module 1/2

data correctly outputted by my module 2/2

Data correctly seen by fdaq (showing only one link)

Nov 4 2019 (64b/66b integrated and TO DOs)

• • 64b/66b quick aurora F/W integrated in FELIX: https://gitlab.cern.ch/atlas-tdaq-felix/firmware/tree/phase2/64b66b

    • locally available in /users/mtrovato/Phase2/64b66b_dacanc/firmware/ 

  • hold slack slightly failed

  • TO DO:

    • implement channel bonding

    • get rid of the vio that reset MGT SM

    • optimize 64b66b for BRAM usage (see email exchange with Marius on Oct 29 2019

    • adapt to FLX-712

    • integrate simulation from Macallan

    • incorporate lpGBT changes as it was done in https://gitlab.cern.ch/atlas-tdaq-felix/firmware/tree/GBTvc709flx712unified

    • remove elastic buffer from MGT

Nov 6 2019 (BRAM breakdown of resource usage 64b/66b)

• • resource usage from aurora in  ~/FELIXaurora4lanes/FELIX_latest_Oct152019_quickaurora/firmware/Projects/felix_fullmode_top/felix_fullmode_top.runs/impl_1/

  • addSOP, split64b: (2+2)x4 = RAMB36

    • split1b uses RAMB18 (to account for that)

• _core, ilas: 7 + 7 + 25 + 101

  • total = 156 (as the number of RAMB36 that I sent to Marius)

Nov 13 2019 (Changes requested for YARR: header in the first byte of the the first 32b word)

• • F/W (FELIX): /users/mtrovato/FELIXaurora4lanes/FELIX_latest_Oct152019_quickaurora_backupOct22

  • F/W (extra VC709): ~/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_headerfirstbytefirst32bword_BackNov13

before fix

after fix

swapped the 32b from RD53a to have header in the first 32b word (N.B: assuming that RD53A spits out header as second 32b word . True in the b frames (1E....32b header data), not obvious from the manual from a frames. 

Nov 19 2019 (SRL16: shft registers with LUTs)

• • LINKS TO BE SORTED OUT:

• https://www.xilinx.com/support/documentation/white_papers/wp271.pdf

• http://www1.pldworld.com/@xilinx/html/technote/TOOL/MANUAL/15i_doc/ALLIANCE/LIB/lib10_10.htm

• https://www.xilinx.com/support/documentation/sw_manuals/xilinx13_2/7series_scm.pdf

• https://www.xilinx.com/support/documentation/application_notes/xapp465.pdf

• https://www.xilinx.com/support/documentation/user_guides/ug474_7Series_CLB.pdf

• https://www.xilinx.com/support/documentation/sw_manuals/xilinx14_1/7series_hdl.pdf

genvar iw;

generate for (iw = 0; iw < 256; iw=iw+1) begin : srl16

SRL16E #(

         .INIT(16'h0000) // Initial Value of Shift Register                                                                                                   

         ) SRL16E_inst (

                        .Q(rx_tdata_shift[iw]), // SRL data output                                                                                            

                        .A0(srl_addr[0]), // Select[0] input                                                                                                  

                        .A1(srl_addr[1]), // Select[1] input                                                                                                  

                        .A2(srl_addr[2]), // Select[2] input                                                                                                  

                        .A3(srl_addr[3]), // Select[3] input                                                                                                  

                        .CE(1'b1), // Clock enable input                                                                                                      

.CLK(clk40_i), // Clock input                                                                                                         

.D(rx_tdata_out[iw]) // SRL data input                                                                                                

                        );

end endgenerate // block: srl16  

Nov 20 2019 (effect of MGT RX elastic buffer on the lane skew and deskew/channel bonding module with no fifos)

• • F/W: 

    • /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_wCB

    • /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_wCB_noelasticbuffer

      • changed in RX_BUF_EN ("FALSE") in _gtx.v

w/ MGT RX elastic buffer: skew change each time I power cycle the chip. Skew as bad as 10x2 40 MHZ clks

no MGT RX elastic buffer

• • deskew / channel bonding module with no fifo: /users/mtrovato/aurora4lanes/aurora4lanesRXonly_woGT_startfromscractch_wCB_noelasticbuffer/aurora4lanesRXonly_woGT_startfromscractch.srcs/sources_1/ip/64b66b_ip/64b66b_deskew.v

Dec 3 2019 (FELIX git repo: TO DO)

• • Merge Phase2/master into Phase2/64b66b

  • resolve conflicts by renaming files of my fullmodewrapper to distinguish with what Kai has

  • commit those files as test files

  • write a testbench

  • coordinate with Marius to write a wrapper the include both RD53B and 64b66b decoder