Data Structures

Top Level

RAD-Gen offers flexible user configuration of tools and provides a common interface for users to pass in thier data and control input parameters for the various modes of operation.

At the highest level, a user can provide either convensional command line (CLI) arguments and/or a YAML / JSON configuration file (using the --top_config_fpath CLI key).

CLI / Structs / Tool Invocation

Each group of command line arguments maps directly to a data structure specified in src/common/data_structs.py. These data structures are responsible for holding all necessary parameters, derived from user inputs, to run the various tools in RAD-Gen.

Command line arguments need to be defined in dedicated structs as they are often used to write scripts, invoke other tools, or perform verification.

So to recap, if a user wanted to define a new tool in RAD-Gen’s suite they would create the following:

  • CLI Struct: e.g. MyToolCLI

  • Data Struct: e.g. MyTool

  • Initialization Function: e.g. init_my_tool_structs(my_tool_conf: dict, common: Common) -> MyTool

  • Invocation: e.g. def run_my_tool(my_tool: MyTool)

        graph LR
   A[User Input] --> B[Initalized Struct];
   B --> C[Tool Invocation];
   C --> D[Results]

In this section, we will focus on the portion of the diagram from User Input to Initalized Struct.

Current Existing Mappings

  • CLI Struct → Tool Data Struct

  • RadGenCLI → Common

  • AsicDseCLI → AsicDSE

  • CoffeCLI → Coffe

  • Ic3dCLI → Ic3d

Format of CLI args, structs, and config file(s)

From src/common/data_structs.py::Common

@dataclass
class Common:
    # ...
    just_config_init: bool = None # flag to determine if the invocation of RAD Gen will just initialze data structures and not run any tools.
    override_outputs: bool = False # If true will override any files which already exist in the output directory
    manual_obj_dir: str = None # If set will use this as the object directory for the current run
    # ...

Corresponding CLI args from src/common/data_structs.py::RadGenCLI:

@dataclass
class RadGenCLI(ParentCLI):
    # ...
    GeneralCLI(key = "override_outputs", shortcut = "-l", datatype = bool, action = "store_true", help_msg = "Uses latest obj / work dir / file paths found in the respective output dirs, overriding existing files"),
    GeneralCLI(key = "manual_obj_dir", shortcut = "-o", datatype = str, help_msg = "Uses user specified obj dir"),
    GeneralCLI(key = "just_config_init", datatype = bool, action = "store_true", help_msg = "Flag to return initialized data structures for whatever subtool is used, without running anything")
    # ...

From the above CLI struct we can pass the command line arguments in a familiar way:

$ python3 rad_gen.py --subtools <subtool_option> --override_outputs --manual_obj_dir path/to/obj_dir --just_config_init

Corresponding rg_top_lvl_conf.yml

# ...
common:
    just_config_init: True
    override_outputs: True
    manual_obj_dir: path/to/obj_dir
# ...

We can see the corresponding keys across the Common struct, the rg_top_lvl_conf.yml file, and command line arguments in the RadGenCLI struct. The general idea is to keep naming consistent across all possible user entry points, ideally avoiding confusion.

Note

Fields in CLI or Tool structs that are initialized to None are not all necessarily optional. Different fields are mandatory for different modes of operation so mandatory ones aren’t always enforced. A nice infrastructure improvement in the future would be to go through all tools and put verify legal options somewhere after struct initialization.

Hierarchically Defined Parameters

Data structures are often nested and have multiple levels of hierarchy, so there is additional logic to deal with this. Let’s go over a slightly more complex example with the asic_dse subtool.

Focusing in on the common.data_structs.CommonAsicFlow data struct contained in common.data_structs.AsicDSE:

From src/common/data_structs.py::AsicDSE:

@dataclass
class AsicDSE:
    # ...
    common_asic_flow: CommonAsicFlow = None # common asic flow settings for all designs
    # ...

From src/common/data_structs.py::CommonAsicFlow:

@dataclass
class CommonAsicFlow:
    # ...
    flow_stages: FlowStages = field(
        default_factory = lambda: FlowStages() # flow stages being run
    )
    # ...

From src/common/data_structs.py::FlowStages:

@dataclass
class FlowStages:
    # ...
    syn: FlowStage = field(
        default_factory = lambda: FlowStage(
            tag = "syn", run = False, tool = "cadence")
    )
    par: FlowStage = field(
        default_factory = lambda: FlowStage(
            tag = "par", run = False, tool = "cadence")
    )
    pt: FlowStage = field(
        default_factory = lambda: FlowStage(
            tag = "pt", run = False, tool = "synopsys")
    )
    # ...

From src/common/data_structs.py::CommonAsicFlow:

@dataclass
class FlowStages:
    # ...
    run: bool = None # Should this stage be run?
    # ...

With respect to above definitions, lets say we want to set the run field within FlowStages at asic_dse.common_asic_flow.flow_stages.syn.run to run only the synthesis stage in a standard cell ASIC flow. The inline code block in the previous sentence is the way to access such a parameter in python syntax, so for simplicity hierarchical CLI params are specified in equivalent syntax.

The hierarhical CLI arg definitions are defined in src/common/data_structs.py::AsicDSECLI:

@dataclass
AsicDseCLI(ParentCLI):
    # ...
    GeneralCLI(key = "common_asic_flow.flow_stages.sram.run", shortcut = "-sram", datatype = bool, action = "store_true", help_msg = "Flag that must be provided if sram macros exist in design (ASIC-DSE)"),
    GeneralCLI(key = "common_asic_flow.flow_stages.syn.run", shortcut = "-syn", datatype = bool, action = "store_true", help_msg = "Flag to run synthesis"),
    GeneralCLI(key = "common_asic_flow.flow_stages.par.run", shortcut = "-par", datatype = bool, action = "store_true", help_msg = "Flag to run place & route"),
    GeneralCLI(key = "common_asic_flow.flow_stages.pt.run", shortcut = "-pt", datatype = bool, action = "store_true", help_msg = "Flag to run primetime (timing & power)"),

We can then manually change the values in this deeply nested struct by passing the data structure hierarhical path as a CLI arg.

$ # To run only synthesis
$ python3 rad_gen.py --top_config_fpath rg_top_lvl_conf.yml --common_asic_flow.flow_stages.sram.run

Initialization

With multiple entry points for parameters to be passed in, a priority is required for cases of overlapping values.

Top Level Priority Order

In order of highest to lowest priority:

  1. Command line arguments

  2. top level config file

  3. CLI default values (defined within CLI structs)

Having such a priority order adds complexity but provides useful functionality for tools using large numbers of user input parameters (most tools in RAD-Gen).

Single Field Example

common/rg_field_init.png

Initialization flow for each field

Once fields have been merged through the priority order, we pass a newly created dictionary into the initialization function for the tool we are running.

Init Function Walkthrough

Regardless of tool or mode of operation, the first initialization function that is called is common.utils.init_structs_top(). Here, the arguments derived from the CLI, config file, and CLI defaults are merged into a single dictionary and passed to the initialization function of the tool being run.

We will continue the walkthrough using asic_dse subtool as an example.

For asic_dse we have the following:

  • CLI Struct: common.data_structs.AsicDseCLI

  • Data Struct: common.data_structs.AsicDSE

  • Initialization Function: common.utils.init_asic_dse_structs()

  • Invocation: asic_dse.run_asic_dse()

common.data_structs.AsicDSE:

@dataclass
class AsicDSE:
    common: Common # common settings for RAD Gen
    mode: AsicDseMode # mode in which asic_dse is running
    stdcell_lib: StdCellLib # technology information for the design
    scripts: ScriptInfo = None # script information for asic_dse
    sweep_conf_fpath: str = None # path to sweep configuration file containing design parameters to sweep
    result_search_path: str = None # path which will look for various output obj directories to parse results from
    common_asic_flow: CommonAsicFlow = None # common asic flow settings for all designs
    asic_flow_settings: HammerFlow = None # asic flow settings for single design
    custom_asic_flow_settings: Dict[str, Any] = None # custom asic flow settings
    design_sweep_info: DesignSweepInfo = None # sweep specific information for a single design
    sram_compiler_settings: SRAMCompilerSettings = None # paths related to SRAM compiler outputs

All of the above fields should be initialized to the values required for the mode of operation specified by the user. The struct containing mode of operation info is common.data_structs.AsicDseMode.

The typical way to intialize a struct field is via the common.utils.init_dataclass() function.

Using the mode field as an example, it would be initialized as follows within common.utils.init_asic_dse_structs():

def init_asic_dse_structs(asic_dse_conf: Dict[str, Any], common: rg_ds.Common) -> rg_ds.AsicDSE:
    # ...
    sweep_conf_valid: bool = asic_dse_conf["sweep_conf_fpath"] != None
    flow_conf_valid: bool = asic_dse_conf["flow_conf_fpaths"] != None

    asic_dse_mode: rg_ds.AsicDseMode = init_dataclass(
        rg_ds.AsicDseMode,
        strip_hier(asic_dse_conf, strip_tag="mode"),
    )
    # Perform post init operations to set fields requiring external / internal dependancies
    asic_dse_mode.init(
        sweep_conf_valid,
        compile_results_flag,
    )
    asic_dse_mode.vlsi.init(
        sweep_conf_valid,
        flow_conf_valid,
        top_lvl_valid,
    )
    # ...

We first call common.utils.init_dataclass() to initialize the mode field, then call the init function within the mode field to set any fields that require additional information to be derived from the user input.

It would be clean if we could simply directly pass the hierarchically defined dictionary key value pairs to the struct, however, during initialization there are a number of fields that have to be derived from the user input, and change depending on the mode of operation or other parameters.

An example of this behavior can be seen in the init function within common.data_structs.AsicDseMode.

def init(
        self,
        sweep_conf_valid: bool,
        compile_results: bool,
) -> None:
    """
        Args:
            sweep_conf_valid: from higher level init function, are preconditions met to run in sweep mode?
            compile_results: is the compile_results flag set?
    """
    # If in sweep mode
    if sweep_conf_valid:
        # If result flat not set we generate sweeps
        if not compile_results:
            self.sweep_gen = True
            self.result_parse = False
        else:
            self.sweep_gen = False
            self.result_parse = True

Which takes control signals derived from user input and sets the sweep_gen and result_parse fields accordingly.

Warning

Users CANNOT define pass in parameters that are initialized within init function.

For example, if a user were to pass in the following CLI arg, an error would be thrown.

$ python3 rad_gen.py --subtools asic_dse --mode.sweep_gen

Internal Init Priority Order

We can define a second stage of priority by which fields are initialized from within a tools initialization function (in previous example common.utils.init_asic_dse_structs()).

In order of highest to lowest priority:

1. Struct internal init functions 1. User Params Merged Dict 2. Field default_factory

Priority examples:

  • Struct internal init functions : e.g. common.data_structs.AsicDseMode.init()

  • User Params Merged Dict: e.g. asic_dse_conf

  • Field default_factory: e.g. vlsi: VLSIMode = field(default_factory = VLSIMode)

Note top two priorities both marked with 1. to denote they are of equal priority and mutually exclusive.

Use Case:

We may be running the COFFE flow with at a particular process technlogy and many additional FPGA architecture parameters. Lets say, for this architecture, we want to evaluate the PPA differences with and without logic block carry chains. Enablement of carry chains is done with a single parameter.

We could duplicate our input configuration file we use, and change the line for the carry chain parameter, but this option is not ideal as it requires manual change and duplication of config files. Instead, if we keep the same base configuration file, and pass in the carry chain parameter as a command line argument, we can easily switch between the two configurations without duplicating files.