Knowledge base schema features¶
Schema file¶
The database schema is defined in the schema.py file in your flow folder. It is evolves together with flow and designed to be easy to reuse and extend. The schema is defined as a set of Python classes that represent the different types of documents in the Qruise KB. Each class is derived from a base class DocumentTemplate and defines the properties of the document as class attributes. The properties can include primitive types (string, integer, float, datetime, boolean), references to other documents or subdocuments. The schema is stored in the database so database can ensure consistency of the data and schema can be discovered and explored by visualization tools.
The schema is stored in the database, allowing the database to ensure consistency of the data, and it can be discovered and explored using visualisation tools.
Documents and subdocuments¶
The Qruise KB stores information as structured documents. Each document represents a physical entity, such as a qubit, resonator, or experiment, and contains a set of key-value pairs that describe its properties.
Documents¶
Documents in Qruise KB are defined as Python classes derived from a DocumentTemplate class. Here's a simple example of a Qubit document. (Real QruiseOS qubits are more complex, but this example is sufficient to illustrate the concept of a document.)
from qruise.kb import DocumentTemplate, LexicalKey
class Qubit(DocumentTemplate):
"""
Qubit document
"""
_key = LexicalKey("name")
name: str
"""
Qubit name i.e Q1
"""
flux: float
"""
Working point flux bias in V
"""
frequency: float
"""
Qubit frequency in Hz at the working point
"""
t1: float
"""
Qubit T1 time in s
"""
The DocumentTemplate class is the base class for all documents in the Qruise KB. The base class along with it's meta-class defined behavior to define, persist and load the objects from and into the database.The properties of the document are defined as class attributes with their types and descriptions. The inline documentation is also a part of schema and stored in the database.
Document key¶
A Document must have a unique identifier, which is used to reference it in the Qruise KB.
The _key attribute defines the unique identifier for the document. In case of Qubit, the LexicalKey type is used to define the key as the name property. The effective database ID of the document is a combination of the name property and the object type, which is Qubit in this case, that would be "Qubit/Q1". Lexical key can combine multiple properties concatenating them with a "+" character. Use properties as type string or int for properties. Float to string conversion can suffer from rounding error.
For named objects, i.e. qubit or a gate the lexical key is a preferred key type.
For a object which is not have individuality i.e. an Experiment one can use a random string key (RandomKey). Alternatively one could use a LexicalKey and initialized it with some pseudo-random value (like cuid, tuid or UUID).
Subdocuments¶
Subdocuments can be used to represent reusable complex data structures. Subdocument can't be inserted on it's own, it must be part of a document. Subdocument can't be referenced by multiple documents. It could be considered as value structured object, so if we want to use same values in another document it need to be copied (which is the default behavior of the Qruise KB Python objects).
An example of a subdocument is a Quantity class, which can be used to represent a physical quantity with its value, unit, default value, and scan range.
from qruise.kb import DocumentTemplate
class Quantity(DocumentTemplate):
"""
Quantity subdocument
"""
_subdocument = ()
value: float | None
"""
Value of the quantity
"""
unit: str | None
"""
Unit of the quantity
"""
default: float | None
"""
Default value of the quantity, i.e. as specified in the datasheet
"""
window: float | None
"""
Range of values to scan around old or default value
"""
The attribute _subdocument is used to mark the class as a subdocument. Subdocuments can't be inserted in the database on their own, they must be part of another document.
Now we can rewrite the Qubit document to use the Quantity subdocument for the values.
from qruise.kb import DocumentTemplate, LexicalKey
class Qubit(DocumentTemplate):
"""
Qubit document
"""
_key = LexicalKey("name")
name: str
"""
Qubit name i.e Q1
"""
flux: Quantity | None
"""
Working point Flux bias in V
"""
frequency: Quantity | None
"""
Qubit frequency in Hz at the working point
"""
t1: Quantity | None
"""
Qubit T1 time in s
"""
t2echo: Quantity | None
"""
Qubit T2 echo time in s
"""
Note that attributes of the Quantity as well as the Qubit class are optional. This is because the Qruise KB is designed to be flexible and allow missing or incomplete data. The None type is used to indicate that a value is not set. If a value is not set. Adding an optional field does not break schema as it not required to have a value, so it can be added at any time.
Properties¶
The properties can be a single value, a collection of values. The type of the single value or a collection item could be a primitive type (string, integer, float, datetime or boolean), reference to another document or a subdocument or set or a list of values.
Datetime¶
It's highly recommended to store datetime in UTC time Zone. Local time is not monotone, and can create issues when accessing the KB from multiple time Zone. UI and tools can convert the UTC time to local time. The Datetime is stored in the database as a string in ISO 8601 format, so it can be easily converted to and from a datetime object.
import datetime
class Experiment(DocumentTemplate):
"""
Experiment document
"""
_key = LexicalKey("id")
id: str
timestamp: datetime.datetime
"""
Experiment record date and time (commit timestamp)
"""
# create an experiment
experiment = Experiment(id="exp1", timestamp=datetime.datetime.now(datetime.timezone.utc)
Optional¶
A property can be marked as optional using either the Optional or the | None type hints. This means that the property can be set to None or not set at all. This is useful for properties that are not always required, or for properties that may not be known at the time of creation. Internally Optional can be considered As a set of at most 1 element. Making a obligatory property optional does not break the schema, as it is not required to have a value. So it can be added at any time. If a value is not set, the property will be None. Adding an optional field does not break schema as it not required to have a value, so it can be added at any time.
Set¶
A property can be marked as a set by using the Set type hint. This means that the property can contain multiple values, but the order of the values is not important. The values in a set are unique, so duplicate values will be removed. The set is unordered collection of unique elements. The set is represented as a Python set object. Adding a set property to a document does not break the schema, as it is will be considered as an empty set. One can also change the type of a property from a single value or an optional value to a set, so it will be interpreted as a set of 0 or 1 elements. Note that Subdocuments are compared by reference so a set of subdocuments can contain multiple identical subdocuments. Adding an sortable property (i.e. name or ordinal) to a subdocument allows to impose order on the set, so implement an ordered set. Set property can't be annotated as Optional. An empty set is equivalent representation of a value not set.
class ResonanceFlux(DocumentTemplate):
_subdocument = ()
biased: Qubit
flux: Quantity
class Coupling(DocumentTemplate):
resonance_fluxes: Set[ResonanceFlux]
List¶
A property can store a list of values. Its list is an ordered collection of values. The values in a list can be of any type, including primitive types (string, integer, float, boolean), references to other documents or subdocuments. The list is represented as a Python list object and stored in a database as a linked list, so requires more resources to store and load, so if order is not important and duplicate values are not to expect set is a preferred way to store multiple values. Adding a list property breaks the schema, since it must be initialized with empty list. The workaround is to add an optional subdocument typed property with a list inside.
class Component(Entity):
_key = LexicalKey("name")
name: str
class ConnectedComponent(Component)
connected: List[Component]
Container types¶
To store experiment data, sampled shapes in the database, the Qruise KB supports bulk numeric types. Qruise KB supports the following bulk types: numpy.ndarray, xarray.Dataset, and xarray.DataArray. To use them just define a property of the type. Numeric property can be marked Optional. Do not use List and Set modifier in combination with numeric types. Use internal dimensions of the array to store multiple values. The numeric types are stored in the database as binary data, so they can be used to store large amounts of data without using a lot of space. xarray.Dataset store multiple values with annotated dimensions metadata about the data, such as the units, axis names. XArray uses Python NETCDF4 (h5netcdf) for serialization. Complex numbers is not part of official NetCDF4 specification, they are supported but is not fully compatible with other NetCDF4 tools. Use separate variables like I and Q for storing readout result. Then storing physical values, it's highly recommended to use standard SI units, like seconds, hertz, volts, etc and annotate data with units. This make display and analysis of data consistent.
import xarray as xr
import datetime
class Experiment(DocumentTemplate):
"""
Experiment document
"""
_key = LexicalKey("id")
id: str
"""
Experiment name i.e Q1
"""
data: xr.Dataset | None
"""
Experiment data
"""
In this example the data property is a reference to an xarray.Dataset object, which is used to store the experiment data. Smaller array (less than 2 kilobytes) are stored inline in the database, larger arrays are stored in Minio object store or in a local filesystem and referenced by a link. The property uses lazy-loading, so the data is not loaded from the object storage until the property it is accessed. This allow effective loading of many documents with subsequent filtering on metadata, without loading the bulk numeric data.
Arbitrary dict¶
The Qruise KB schema supports properties of type dict, allowing users to store arbitrary dictionaries containing structured key-value data. These dictionaries are serialised using hjson, a human-readable format that supports standard types as well as numpy arrays.
A typical use case is storing third-party objects that can be represented as dictionaries. Here's an example showing how Qiskit discriminators can be stored in the KB and included within a readout resonator:
class ReadoutDiscriminator(DocumentTemplate):
"""
Qiskit discriminator object
"""
_subdocument = ()
number_of_states: int | None
discriminator_type: str
discriminator_config: dict | None
assignment_matrix: List[float]
class ReadoutResonator(LineComponent):
discriminator: ReadoutDiscriminator | None
discriminator_3_state: ReadoutDiscriminator | None
It's possible to store any object in the database as a dict, but it's not recommended, since effectively it will be stored as a string in the database and it will be difficult to access the properties of the object without custom Python code. Database will not be able to check schema and consistency of the data. So it's better to use a subdocument or a document with properties to store the data the structure you are in control of.
Methods¶
Usually document template classes are pure data classes, that is they don't have any application specific methods except property getter/setters. Nevertheless, it is possible to define methods in the document template class. The methods can be used to add a behavior to a class. The methods can be defined as normal Python methods and can use the self parameter to access the document properties.
class Fit(DocumentTemplate):
"""Represents a fitting model"""
_abstract=()
_subdocument=()
@abstract
def __call__(self, x:float):
"""
Fit the data
"""
pass
class ParabolicFit(Fit):
"""
Parabolic fit (i.e. frequency vs flux)
"""
a: float
x0: float
y0: float
def __call__(self, x: float) -> float:
return self.a * (x - self.x0) ** 2 + self.y0
The Fit class is an abstract class that defines the interface for fitting models. The __call__ method is defined as an abstract method, which means that it must be implemented by any subclass of the Fit class.
Class Fit is marked as abstract by the _abstract attribute. The ParabolicFit class is a concrete implementation of the Fit class that implements the __call__ method. The __call__ method can be used to fit the data and return the fitted value. The ParabolicFit class can be used as a property of another document, just like any other document. Here is an example of how to use the Fit class as a property of a document:
The freq_of_flux property is a reference to a Fit object, which can be used to fit the data. The __call__ method of the Fit class can be used to fit the data and return the fitted value. The ParabolicFit is a specific implementation of the Fit class that implements the __call__ method. Storing / loading an object from or to the database takes care about instantiating the correct concrete class. This allows using different model implementations without changing the schema.
Document inheritance¶
As in Python the Qruise KB supports document type inheritance. This means that you can create a new document class that inherits from an existing document class and extends it by adds properties and/or methods. This is useful for creating specialized versions of documents or for reusing common properties across multiple documents. The QruiseKB allows fetching documents can be based on type, so having a specific base class can be useful to impose specific load behavior. In particular, standard Qruise KB schema has base class "Entity" which is used to mark documents keeping QPU characterization values and and effectively needed for system setup.
from qruise.kb import DocumentTemplate, LexicalKey, Entity
class Entity(DocumentTemplate):
"""
Entity document
"""
_abstract = ()
class Component(Entity):
"""
Component document
"""
_key = LexicalKey("name")
name: str
"""
Component name i.e Q1
"""
class Qubit(Component):
"""
Qubit document
"""
freq: Quantity | None
"""
Qubit frequency in Hz at the working point
"""
t1: Quantity | None
"""
Qubit T1 time in s
"""
t2echo: Quantity | None
"""
Qubit T2 echo time in s
"""
readout: Readout | None
"""
Readout resonator
"""
class Readout(Component):
"""
freq: Quantity | None
"""
Readout resonator frequency in Hz at the working point
amplitude: Quantity | None
"""
readout amplitude
"""
duration: Quantity | None
"""
Readout duration in s
"""
qubit: Qubit | None
"""
Deriving the Qubit class from the Component class allows us to reuse the name property and the _key attribute. The Readout class is also derived from the Component class, so it can be used as a property of the Qubit class. The readout property is a reference to another document, which is a Readout document. This allows us to connect qubit , amplitude and duration as part of the qubit document.
The class Component is derived from the Entity class, which is a base class for all documents that represent physical entities. The Entity class can be used to mark documents that keep QPU characterization, so they can be loaded with a single database API call.
The Entity class is marked as anstract with a class variable _abstract=() which prevents instantiation of the class.
Another examples of base classes are:
class Eventwhich is a base for logged events like Experiments or manual correction.class Calibrationwhich is a base for storing calibrated control sequences like quantum gates or incoherent operations like unconditional reset.
Schema inheritance¶
To enable correct functioning of dashboard as also to follow best practices it's recommended that all Qruise-OS KB schemas have same set of basis classes. So Qruise-OS defines a core schema, and all Qruise-OS schemas should be derived from it. qruise experiment library defines types representing standard characterization and calibration experiments, so including it in the schema reduce a lot of boilerplate code and allows schema evolution with new releases of the Qruise-OS.
Here is an example of a minimal schema.py file using core and experiment catalog schemas.
from typing import List, Optional, Sequence, Set
from qruise.kb import DocumentTemplate, LexicalKey
from qruise.experiment.schema.core import (
CoreSchema,
Component,
ConnectedComponent,
Entity,
Fit,
Quantity,
QruiseObject,
QuantityList,
Qubit,
)
from qruise.experiment.schema.experiments import ( # noqa: F401, F811
Ramsey,
Ramsey12,
T2StarRamsey,
T1,
T2StarRamsey,
ReadoutDiscriminator,
ReadoutDiscriminator2,
)
TITLE = "Example"
DESCRIPTION = "Schema for QruiseOS flow examples"
AUTHORS = [
"contact@qruise.com",
]
with CoreSchema(
title=TITLE,
description=DESCRIPTION,
authors=AUTHORS,
) as ExampleSchema:
class ParabolicFit(Fit):
a: Quantity
x0: Quantity
y0: Quantity
def __call__(self, x):
return self.a * (x - self.x0) ** 2 + self.y0
class FluxCrosstalk(Entity):
name: str
_key = LexicalKey("name")
target: "Component"
biased: "Component"
crosstalk: Quantity
class ReadoutLine(QruiseObject):
_key = LexicalKey("name")
index: int
lo_freq: Optional[Quantity] # Local oscillator frequency in Hertz
class Resonator(Component):
freq: Optional[Quantity]
kappa: Optional[Quantity]
class ReadoutResonator(ConnectedComponent):
freq: Optional[Quantity]
resonator: Optional[Resonator]
amplitude: Optional[Quantity]
amplitude_spectroscopy: Optional[Quantity] # Amplitude of the readout scan
relax_time: Optional[Quantity]
duration: Optional[Quantity]
delay: Optional[Quantity]
line: Optional[ReadoutLine]
inverse: Optional[bool]
discriminator: Optional["ReadoutDiscriminator"]
discriminator_3_state: Optional["ReadoutDiscriminator"]
dispersive_shift: Optional[Quantity]
class DragPulse(DocumentTemplate):
_subdocument = ()
amplitude: Optional[
Quantity
] # Amplitude of a default gaussian (6-sigma) PI-pulse in volts, specific to QC implementation
duration: Optional[
Quantity
] # Duration of a default gaussian (6-sigma) PI-pulse in seconds, specific to QC implementation
drag: Optional[Quantity]
freq: Optional[Quantity] # Qubit frequency in hertz, adjusted.
drag_optimal_control: Optional[Quantity] # delta, adjusted.
class Qubit(Qubit):
"""
Qubit component
Attributes
----------
flux : Optional[Quantity]
Flux line signal level in volts
freq_coarse : Optional[Quantity]
Coarse spectroscopy measured qubit frequency in hertz from qubit spectroscopy.
freq : Optional[Quantity]
Ramsey measured accurate qubit frequency in hertz, measured with with a Ramsey experiment.
anhar_coarse : Optional[Quantity]
Coarse estimated anharmonicity, measured by pulsed spectroscopy of the second transition (Hz)
anhar : Optional[Quantity]
Anharmonicity, measured by Ramsey of the second transition (Hz)
t1 : Optional[Quantity]
:math:`T1` time in seconds, measured by measuring the decay of the excited state in Spin-echo experiment
t2star : Optional[Quantity]
:math:`T2\\star` time in seconds, measured by measuring the decay of the state in Ramsey experiment (s)
"""
readout: Optional["ReadoutResonator"]
freq_specification: Optional[
Quantity
] # Qubit frequency in Hertz, from chip specification
lo_freq: Optional[Quantity] # Local oscillator frequency in Hertz
freq_of_flux: Optional[Fit]
flux_crosstalks: Set[FluxCrosstalk]
flux: Optional[
Quantity
] # Working point (coupling off) of the flux line in volts
freq_coarse: Optional[
Quantity
] # Qubit frequency, measured with a pulsed spectroscopy. (Hz)
freq: Optional[
Quantity
] # Qubit frequency, precisely measured with a Ramsey experiment. (Hz)
anhar_coarse: Optional[
Quantity
] # Anharmonicity, measured with a pulsed spectroscopy (Hz)
anhar: Optional[
Quantity
] # Anharmonicity, precisely measured with a Ramsey experiment (Hz)
t1: Optional[
Quantity
] # :math:`T1` time in seconds, measured by measuring the decay of the excited state in Spin-echo experiment
t2star: Optional[
Quantity
] # :math:`T2\star` time in seconds, measured by measuring the decay of the state in Ramsey experiment (s)
x180_amplitude_spectroscopy: Optional[
Quantity
] # CW pulse amplitude for the spectroscopy scan
x180: Optional[DragPulse]
x90: Optional[DragPulse]
x180_ef: Optional[DragPulse]
class Coupling(ConnectedComponent):
"""Coupling component"""
strength: Optional[Quantity]
resonance_fluxes: Set[ResonanceFlux]
flux_off: Optional[Quantity]
@classmethod
def find(cls, connected: Sequence[Component]):
return next(
iter(
coupling
for coupling in Coupling.get_instances()
if connected == coupling.connected
),
None,
)
# Experiments
class Ramsey(Ramsey):
pass
class Ramsey12(Ramsey12):
pass
class T2StarRamsey(T2StarRamsey):
pass
class T1(T1):
pass
class ReadoutDiscriminator2(ReadoutDiscriminator2):
pass
CoreSchema imported from qruise-experiment library contains boilerplate components for a consistent Qruise schema. Invoking it with CoreShema effectively creates a new scheme where all elements of parent schema are included. Open the context with with statement allows defining, extending and importing new document type in the context of the schema.
The documents defined in the base schema can be extended by sub-classing it with the same name like class Qubit(Qubit): All document type in a schema exist in one namespace, so deriving a class with a same name, effectively extends the bases one by adding a properties to it. If a base class (like in this case T1) does not exist in the current schema the type will be imported including all types it transitively relies on. So only types explicitly subclassed in schema context imported from qruise.experiment.schema.experiments will be included in the schema. This allows creating a schema containing only the types and properties only needed to store results of used experiments and can be easily extended by importing new types from library schemas.
Database schema merge¶
A schema evolves as the knowledge on QPU evolves. We add experiments, measuring or learning additional parameters. The schema is stored in the database but in the same time is defined a python So it can happen that schema stored in KB and the schema defined in schema.py are not identical. Nevertheless if changes are compatible issue can be resolved by mixing both schemas. The schema used in the script or notebook would be dynamic combination of both, thus allowing load/modify/store cycle in an experiment script or a notebook. Everything except methods is stored in in the KB, so one can start access and modify the KB without having a schema.py file.
Schema update¶
Schema update part of natural flow development. Every time we introduce a new experiment, new measured we have to add a property or document types. Extending schema document values and enable proper interaction with other components of Qruise-OS (dashboard, datas export and analysis tools). Schema can be considered as set of constrains on your stored data. In what aspect weakening that constrains introduce a non breaking change. Here some examples of safe schema modification operations.
- Adding a new document class.
- Modifying or removing a document class, if no instance of it exists in the KB.
- Adding a Optional or Set property (cardinality 0) to an existing documents.
- Deleting a Optional property where no instance of it exists in the KB.
- Marking an obligatory property as optional.
- Changing an optional property to a set property.
- Changing inheritance hierarchy, if it does introduce new mandatory properties in any existing documents.
- Modifying inline documentation of a property or document class.
So for some operation data modification and multiples steps are needed before a schema change can be applied. For example to delete a obligatory property, it must be first changed to optional, than set to None in all instances and then deleted.
Schema update with CLI tool¶
Assuming one have .QKB.json and schema.py file in the current directory, one can use the qruise CLI tool to update the schema. The tool will load the schema from the KB and merge it with the schema defined in the schema.py file. The merged schema will be in qruise KB.
Schema update Python API¶
The schema update notebook is usually the first step of the flow to ensure the KB schema will be able to store the data and KB objects have all expected properties. Qruise KB API provide methods to simplify migration.
from qruise.kb import Schema, connect
from pprint import pprint
client = connect()
schema = Schema()
# read old schema as list of dictionaries
old_schema = client.get_schema()
# Update schema meta and update from `schema.py` and then show all classes defined in it. replace=True enables deletion of unused document types removed from schema
migration = schema.prepare_migration(
old_schema=old_schema,
module="schema",
replace=True,
)
# print summary of the migration
pprint(migration.get_diff())
# apply migration if needed
if migration.is_pending():
migration.commit(client, commit_msg="Update schema")
Qruise KB Session¶
In a typical experiment notebook the schema object does not need to be instantiated explicitly.
from qruise.kb import create_session
from pprint import pprint
session = create_session()
session.load_schema() # load schema from the database
session.import_schema() # load schema from the schema.py file
pprint(session.schema.types)
Main components of a Session are:
client, API to access TerminusDB databaseschema, schema object to access the schemastore, API to access blob object storage (Minio or local filesystem)blob_store, helper to store and load numeric data objects
The create_session function use .QKB.json from the current directory to select the database to connect to. The load_schema method loads the schema from the database.
session.import_schema loads schema.py file and merges it with the schema in the database. The import_schema method will not modify the database schema. To persist the changes, the session.save_schema method must be called. It's better to use schema update procedure described above since it checks if the schema is really changed and helps visualize the changes.