Analysis & report¶
Analysis¶
analysis ¶
Statistical analysis: OLS, ANOVA, lack-of-fit, mixed models.
EstimatorBackend ¶
Bases: Protocol
Minimal OLS backend seam (statsmodels today; mockable in tests).
fit_ols ¶
Return a result object with params, bse, tvalues, pvalues, etc.
FitResult
dataclass
¶
FitResult(
names,
coef,
se,
tvalues,
pvalues,
resid,
sigma2,
r_squared,
dof,
fitted=None,
r_squared_adj=float("nan"),
leverage=None,
studentized_resid=None,
cooks_distance=None,
cov_type="nonrobust",
blocks=None,
_sm_result=None,
)
Result of an OLS fit, with anomaly diagnostics.
Container for coefficient estimates, inferential statistics and per-run
influence measures. Serialize with :meth:to_dict / :meth:from_dict for
handoff to reports and agents.
Formulas
- R²:
1 - RSS / TSSwhereTSS = sum((y - ybar)^2). - Adjusted R²:
1 - (1 - R²) * (N - 1) / dof. - Residual variance:
sigma2 = RSS / dof.
Attributes:
| Name | Type | Description |
|---|---|---|
names |
list of str
|
Column (term) names of the model matrix (including block dummies). |
coef, se, tvalues, pvalues |
ndarray
|
Coefficient estimates, standard errors, t statistics and p-values. |
resid |
ndarray
|
Residuals |
sigma2 |
float
|
Residual variance estimate |
r_squared, r_squared_adj |
float
|
Coefficient of determination and its adjusted version. |
dof |
int
|
Residual degrees of freedom |
fitted |
(ndarray, optional)
|
Fitted values. |
leverage |
(ndarray, optional)
|
Hat-matrix diagonal |
studentized_resid |
(ndarray, optional)
|
Externally studentized (deleted) residuals. |
cooks_distance |
(ndarray, optional)
|
Cook's distance (influence) of each run. |
cov_type |
str
|
Covariance estimator used ( |
blocks |
str or None
|
Block column name, |
anomalies ¶
Runs flagged as atypical or influential, with the reason.
Combines three standard influence rules tuned for small DoE designs. Returns an empty frame when leverage / studentized residuals were not computed.
Formulas
- Outlier:
|r_i^*| > 3. - High leverage:
h_ii > 2p / N. - Influential:
Cook's D_i > 1(Cook & Weisberg absolute cutoff; the4/Nrule over-flags deliberately high-leverage DoE points).
Returns:
| Type | Description |
|---|---|
DataFrame
|
Columns |
MixedFitResult
dataclass
¶
MixedFitResult(
names,
coef,
se,
zvalues,
pvalues,
resid,
sigma2,
groups,
n_groups,
re_var,
method="reml",
converged=True,
llf=float("nan"),
aic=float("nan"),
bic=float("nan"),
fitted=None,
_sm_result=None,
)
Result of a linear mixed model (REML / ML) fit.
Fixed-effect estimates with Wald statistics plus random-effect variance
components from statsmodels MixedLM.
Formulas
- Wald z:
z_j = beta_j / SE(beta_j),p = 2 * Phi(-|z_j|). - Information criteria: AIC/BIC from the fitted mixed-model likelihood.
Attributes:
| Name | Type | Description |
|---|---|---|
names |
list of str
|
Fixed-effect term names. |
coef, se, zvalues, pvalues |
ndarray
|
Fixed-effect estimates and Wald statistics. |
resid |
ndarray
|
Residuals (response minus fixed+random fitted values when available). |
sigma2 |
float
|
Residual (within-group) variance. |
groups |
str
|
Name of the grouping factor (or |
n_groups |
int
|
Number of groups. |
re_var |
dict
|
Random-effect variance components (label -> variance). |
method |
{'reml', 'ml'}
|
Estimation method. |
converged |
bool
|
Whether the optimizer reported convergence. |
llf, aic, bic |
float
|
Log-likelihood and information criteria. |
fitted |
(ndarray, optional)
|
Fitted values. |
anova_table ¶
Partial-F (Wald) ANOVA table for an OLS :class:FitResult.
For single-degree-of-freedom terms (the usual coded DoE case) each row is a
partial F-test of that term given all others. Multi-df categorical blocks
appear as separate dummy rows (block[...]).
Formulas
- Single-df term:
F_j = t_j^2wheret_jis the OLS t-statistic (equivalent to a Type III partial F when terms are orthogonal). - p-value:
P(F_{1, dof} > F_j)from the residual dof of the fit.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fit
|
FitResult
|
Result of :func: |
required |
typ
|
(2, 3, 'II', 'III')
|
Accepted for API compatibility; the table is a partial-F / Wald table (Type III for single-df terms). A formula-based statsmodels Type II/III table is used when the underlying result was fit with a formula. |
2
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Columns |
Examples:
attach_blocks ¶
Return a copy of design with a block column and metadata['blocking'].
Writes per-run block labels into the design matrix and records the blocking
metadata so :func:fit_linear_model picks up the column automatically.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Source design. |
required |
blocks
|
array - like
|
Per-run block labels (length |
required |
name
|
str
|
Column name for the block factor. |
"block"
|
Returns:
| Type | Description |
|---|---|
Design
|
New design with the block column appended (or overwritten). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
fit_linear_model ¶
Fit an OLS linear model to a design's responses.
Ordinary least squares via statsmodels on the resolved model matrix X.
Factor columns come from the design matrix; block/group columns are excluded
from the default main-effects model. Influence diagnostics (leverage,
studentized residuals, Cook's distance) use the classical hat matrix.
Formulas
- Coefficients:
beta = (X'X)^-1 X'y. - Residual variance:
sigma2 = RSS / (N - p)withRSS = sum((y - X @ beta)^2). - Leverage:
h_ii = x_i' (X'X)^-1 x_i. - Studentized residual:
r_i^* = r_i / sqrt(sigma2 * (1 - h_ii))(externally studentized when residual dof permits). - Cook's D:
D_i = (r_i^2 / (p * sigma2)) * h_ii / (1 - h_ii)^2.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
The executed design providing the factor levels. |
required |
response
|
array - like
|
Measured response per run. |
required |
model
|
Model
|
Model to fit; taken from |
None
|
blocks
|
(None, False, str or array - like)
|
Fixed block factor: a column name in |
None
|
cov_type
|
('nonrobust', 'HC0', 'HC1', 'HC3')
|
Covariance estimator for standard errors (via statsmodels). |
"nonrobust"
|
report
|
(None, bool, str, Path or dict)
|
If not |
None
|
Returns:
| Type | Description |
|---|---|
FitResult
|
The fitted model with coefficients, statistics and anomaly diagnostics. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
fit_mixed_model ¶
Fit a linear mixed model (random intercept / slopes via MixedLM).
Fixed effects are fit on the same resolved model matrix as OLS; a grouping
factor introduces random intercepts (or slopes via re_formula). Typical
use: batch, whole-plot, or hard-to-change factors in split-plot designs.
Wald z-tests and p-values are reported for fixed effects.
Formulas
- Model:
y = X @ beta + Z @ u + epsilonwith random effectsu ~ N(0, G)andepsilon ~ N(0, sigma2 I). - REML: maximize the restricted likelihood (variance components + fixed effects given those variances); ML uses the full likelihood instead.
- Wald test:
z = beta_j / SE(beta_j), two-sided normal p-value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Executed design. |
required |
response
|
array - like
|
Measured response per run. |
required |
groups
|
str or array - like
|
Grouping factor for random effects (column name or per-run labels). Typical use: batch, whole-plot, or hard-to-change factor. |
required |
model
|
Model
|
Fixed-effects model; defaults as in :func: |
None
|
re_formula
|
str
|
Random-effects formula in statsmodels style ( |
"1"
|
method
|
('reml', 'ml')
|
Estimation method (ignored if |
"reml"
|
reml
|
bool
|
If set, overrides |
None
|
Returns:
| Type | Description |
|---|---|
MixedFitResult
|
Fixed effects, variance components and fit diagnostics. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If groups are invalid (fewer than 2 levels, length mismatch, etc.). |
Examples:
half_normal_data ¶
Half-normal quantiles vs. effect magnitude, sorted.
Builds the data for a Daniel half-normal plot: inactive effects fall on a straight line through the origin; active effects depart upward.
Formulas
For rank k = 1..m (sorted by |effect|):
q_k = Phi^-1(0.5 + 0.5 * (k - 0.5) / m)
where Phi is the standard normal CDF.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
effects
|
array - like
|
Estimated effects (or coefficients). |
required |
labels
|
sequence of str
|
Labels for each effect; defaults to |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Columns |
Examples:
lack_of_fit ¶
Lack-of-fit vs pure-error decomposition when replicate runs exist.
Identical rows of the (non-block) factor matrix are treated as replicates. Pure-error SS is the within-replicate variation; lack-of-fit SS is what remains of the model RSS after subtracting pure error.
Formulas
- Pure error:
SS_PE = sum_g sum_{i in g} (y_i - ybar_g)^2,df_PE = sum_g (n_g - 1)over replicate groupsg. - Lack of fit:
SS_LOF = RSS - SS_PE,df_LOF = dof - df_PE. - F-test:
F = MS_LOF / MS_PEwithMS = SS / df;p = P(F_{df_LOF, df_PE} > F).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Executed design. |
required |
response
|
array - like
|
Measured response per run. |
required |
model
|
Model
|
Model used for the fitted RSS; defaults as in :func: |
None
|
blocks
|
str or array - like
|
Optional fixed blocks (same semantics as :func: |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
One-row-per-source table with columns |
Raises:
| Type | Description |
|---|---|
ValueError
|
If there are no replicate groups (no pure-error degrees of freedom). |
Examples:
>>> import doekit as ed
>>> import pandas as pd
>>> fac = ed.full_factorial({"A": [-1, 1], "B": [-1, 1]})
>>> d = ed.Design(matrix=pd.concat([fac.matrix, fac.matrix], ignore_index=True),
... factors=list(fac.factors))
>>> lof = ed.lack_of_fit(d, (fac.matrix["A"] + fac.matrix["B"]).tolist() * 2)
>>> "pure_error" in set(lof["source"])
True
main_effects ¶
Estimated main effects of a screening design.
Fits a main-effects model (no intercept by default) and returns one magnitude per term. The relative ordering is identical under both scales, so half-normal screening is unchanged.
Formulas
- Coefficient scale:
beta_jfromy ~ sum_j beta_j x_j(no intercept). - Effect scale (Montgomery):
E_j = mean(y | x_j = +1) - mean(y | x_j = -1). For an orthogonal factor coded+/-1,E_j = 2 * beta_j.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
The executed design. |
required |
response
|
array - like
|
Measured response per run. |
required |
model
|
Model
|
Model to fit; a main-effects model (no intercept) is used if omitted. |
None
|
scale
|
('coefficient', 'effect')
|
Magnitude convention:
|
"coefficient"
|
Returns:
| Type | Description |
|---|---|
Series
|
Effects indexed by term name. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
Design advisor¶
advise ¶
Advisory / recommendation policies.
ExperimentHistory ¶
A small, similarity-searchable collection of :class:ExperimentRecord.
Typically built from an :class:~doekit.presentation.workspace.ExperimentProject
via :meth:from_project.
Examples:
>>> import doekit as ed
>>> hist = ed.ExperimentHistory([
... ed.ExperimentRecord("w1", "screening", ["A", "B"], {}),
... ])
>>> len(hist) == 1
True
find_similar ¶
Rank records by objective match and factor overlap.
Formulas
score = 0.6 * objective_match + 0.4 * |factors ∩ target| / |target|.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective
|
str
|
Goal label for the new case. |
required |
factor_names
|
sequence of str
|
Factor names in the new case. |
required |
top_k
|
int
|
Maximum records to return. |
5
|
Returns:
| Type | Description |
|---|---|
list of ExperimentRecord
|
Similar records, best match first. |
Examples:
from_project
classmethod
¶
Build history from an :class:ExperimentProject's waves (best-effort).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
project
|
ExperimentProject
|
Traceable project whose waves are scanned. |
required |
Returns:
| Type | Description |
|---|---|
ExperimentHistory
|
History with one record per readable wave. |
Examples:
ExperimentRecord
dataclass
¶
A compact record of a past experiment for similarity search.
Attributes:
| Name | Type | Description |
|---|---|---|
experiment_id |
str
|
Stable identifier (e.g. a wave id). |
objective |
str
|
Goal label ( |
factor_names |
list of str
|
Factor names involved. |
metrics |
dict
|
Outcome signals ( |
metadata |
dict
|
Provenance and extra context. |
Examples:
>>> import doekit as ed
>>> rec = ed.ExperimentRecord("w1", "screening", ["A", "B"], {"D_efficiency": 85.0})
>>> rec.objective == "screening"
True
HistoricalRecommendation
dataclass
¶
Actionable advice derived from similar experiment history.
Attributes:
| Name | Type | Description |
|---|---|---|
title |
str
|
Short headline for the recommendation. |
rationale |
str
|
Why this advice follows from past experiments. |
actions |
list of str
|
Concrete next steps for the experimenter. |
Examples:
>>> import doekit as ed
>>> hist = ed.ExperimentHistory()
>>> advice = ed.historical_recommendation(hist, "optimization", ["T", "pH"])
>>> len(advice.actions) >= 1
True
PriorEstimate
dataclass
¶
PriorEstimate(
objective,
n_sources,
expected_delta_d_efficiency=0.0,
expected_delta_mean_power=0.0,
expected_uncertainty=0.5,
metadata=dict(),
)
Priors transferred from similar past experiments.
Attributes:
| Name | Type | Description |
|---|---|---|
objective |
str
|
Goal label the priors apply to. |
n_sources |
int
|
Number of similar records averaged. |
expected_delta_d_efficiency |
float
|
Mean historical D-efficiency gain. |
expected_delta_mean_power |
float
|
Mean historical power gain. |
expected_uncertainty |
float
|
Mean historical uncertainty level. |
metadata |
dict
|
e.g. |
Examples:
>>> import doekit as ed
>>> hist = ed.ExperimentHistory()
>>> prior = ed.learn_priors(hist, "screening", ["A", "B"])
>>> prior.n_sources == 0
True
Recommendation
dataclass
¶
Result of :func:recommend_design.
Attributes:
| Name | Type | Description |
|---|---|---|
method |
str
|
Winning design method label from the shortlist. |
design |
Design
|
Recommended experimental design. |
model |
Model
|
Model used to evaluate and attach to the design. |
rationale |
str
|
Human-readable justification. |
table |
DataFrame
|
Evaluated alternatives (runs, D/G-eff, SPV, feasibility flags). |
caveats |
list of str
|
Assumptions and limitations. |
scenario |
dict
|
Goal, factor count, budget, constraints snapshot. |
Examples:
>>> import doekit as ed
>>> rec = ed.recommend_design("screening", factors=4, budget=20, seed=0)
>>> rec.design.n_runs <= 20
True
historical_recommendation ¶
Turn similar history into a short strategy suggestion for a new case.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
history
|
ExperimentHistory
|
Past experiment records. |
required |
objective
|
str
|
Goal label for the new case. |
required |
factor_names
|
sequence of str
|
Factor names in the new case. |
required |
top_k
|
int
|
Number of similar records to consult. |
5
|
Returns:
| Type | Description |
|---|---|
HistoricalRecommendation
|
Title, rationale, and actionable steps. |
Notes
When mean historical delta_D_efficiency >= 5, the advice favours
expansion; otherwise it suggests caution and model refinement.
Examples:
learn_priors ¶
Average outcome signals over similar past experiments into priors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
history
|
ExperimentHistory
|
Past experiment records. |
required |
objective
|
str
|
Goal label for the new case. |
required |
factor_names
|
sequence of str
|
Factor names in the new case. |
required |
top_k
|
int
|
Number of similar records to average. |
5
|
Returns:
| Type | Description |
|---|---|
PriorEstimate
|
Expected deltas and uncertainty; |
Examples:
recommend_design ¶
recommend_design(
goal,
factors,
budget=None,
model_order=None,
priorities=None,
constrained=False,
constraints=None,
mixture=False,
hard_to_change=None,
effect_size=1.0,
sigma=1.0,
seed=None,
n_region=4000,
)
Recommend the best experimental-design method for a given case.
Transparent advisor: rule-based shortlist, then multi-objective ranking
with D/A/G-efficiency metrics under user priorities.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
goal
|
('screening', 'optimization')
|
Experimental objective. |
"screening"
|
factors
|
int or dict or sequence of Factor
|
Factor specification (count or explicit factors). |
required |
budget
|
int
|
Maximum number of runs. |
None
|
model_order
|
('linear', 'interactions', 'quadratic')
|
Assumed model order; defaults to linear (screening) or quadratic (optimization). |
"linear"
|
priorities
|
dict
|
Weights for |
None
|
constrained
|
bool
|
Deprecated. Use |
False
|
constraints
|
Constraints or dict
|
Native constraints (mixture, hard_to_change, irregular, run_cost, …). |
None
|
mixture
|
bool
|
Shortcut for mixture / simplex shortlist. |
False
|
hard_to_change
|
sequence of str
|
Shortcut for split-plot whole-plot factor names. |
None
|
effect_size
|
float
|
Anticipated effect size (documented in caveats for power). |
1.0
|
sigma
|
float
|
Noise standard deviation (documented in caveats). |
1.0
|
seed
|
int
|
RNG seed for optimal-design search and region sampling. |
None
|
n_region
|
int
|
Region sample size for G-efficiency evaluation. |
4000
|
Returns:
| Type | Description |
|---|---|
Recommendation
|
Winning method, design, model, rationale, and alternatives table. |
Raises:
| Type | Description |
|---|---|
InapplicableDesign
|
When no catalog design applies to the factor / constraint combination. |
ValueError
|
For unknown |
Notes
"Best" is a multi-objective trade-off — adjust priorities for your
case. After the first wave, prefer :func:~doekit.propose_next_runs for
sequential augmentation.
Examples:
Experiment aggregate¶
experiment ¶
Experiment aggregate (product contract).
Experiment
dataclass
¶
Experiment(
design,
model=None,
response_names=(lambda: ["y"])(),
responses=None,
evaluation=None,
fits=dict(),
recommendation=None,
metadata=dict(),
response=None,
fit=None,
)
End-to-end experiment handle: design through report and persistence.
Stateful aggregate composing recommend, evaluate, ingest, sequential propose, decision, and export without embedding presentation side-effects in domain functions.
Attributes:
| Name | Type | Description |
|---|---|---|
design |
Design
|
Current experimental design. |
model |
(Model, optional)
|
Fitted model specification. |
response_names |
list of str
|
Names of response columns. |
responses |
(DataFrame, optional)
|
Multi-column ingested responses. |
evaluation |
(DesignEvaluation, optional)
|
Cached design-quality evaluation. |
fits |
dict
|
Per-response :class: |
recommendation |
(Recommendation, optional)
|
Advisor result when built via :meth: |
metadata |
dict
|
Goal, budget, and other experiment context. |
response |
(ndarray, optional)
|
Primary (first) response vector for sequential / report. |
fit |
(FitResult, optional)
|
Fit for the primary response. |
Examples:
>>> import doekit as ed
>>> exp = ed.experiment(goal="screening", factors=4, budget=12)
>>> ev = exp.evaluate(seed=0)
>>> ev.d_efficiency > 0
True
compare ¶
Ask whether n_add more runs are worth it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_add
|
int
|
Proposed augmentation size. |
4
|
**kwargs
|
Forwarded to :meth: |
{}
|
Returns:
| Type | Description |
|---|---|
DesignComparison
|
Metric deltas and |
Examples:
conclude ¶
Write automatic conclusions into a wave (semantic handoff artifact).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
wave
|
Wave, str, or Path
|
Target wave directory. |
required |
thresholds
|
mapping
|
Quality thresholds for conclusions. |
None
|
lang
|
str
|
Language for narrative text. |
"en"
|
write_html
|
bool
|
Emit an HTML conclusions page. |
False
|
comparison
|
DesignComparison
|
Comparison artifact to include. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Conclusions payload written by the wave. |
decide_next ¶
decide_next(
n_add=4,
*,
intent="learn",
budget=None,
risk_tolerance="moderate",
proposal=None,
use_calibration=False,
history=None,
scorer=None,
policy=None,
**kwargs,
)
Decide the next action (stop / augment / refine / redesign).
Proposes the next batch (unless proposal is given) and feeds its
signals to the decision engine — comparison deltas + worth_it for
learn, predicted_improvement / explore_exploit for
optimize — together with the run budget and design quality. When
history is given, convergence is checked and can force a stop.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_add
|
int
|
Runs to propose when |
4
|
intent
|
('learn', 'optimize')
|
Passed to :meth: |
"learn"
|
budget
|
int
|
Total run budget; falls back to |
None
|
risk_tolerance
|
('low', 'moderate', 'high')
|
Decision policy sensitivity. |
"low"
|
proposal
|
NextRunsProposal
|
Precomputed proposal; :meth: |
None
|
use_calibration
|
bool
|
Use surrogate calibration for optimize uncertainty. |
False
|
history
|
iterable
|
Per-generation values for :func: |
None
|
scorer
|
Custom :class: |
None
|
|
policy
|
Custom :class: |
None
|
|
**kwargs
|
Forwarded to :meth: |
{}
|
Returns:
| Type | Description |
|---|---|
Decision
|
Recommended action with diagnostics in |
Examples:
desirability ¶
Overall desirability across ingested multi-response data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
goals
|
mapping
|
|
None
|
Returns:
| Type | Description |
|---|---|
Series
|
Per-run desirability from :func: |
Raises:
| Type | Description |
|---|---|
ValueError
|
If :meth: |
Examples:
evaluate ¶
Run design-quality evaluation and cache the result.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
DesignEvaluation
|
Cached on |
Examples:
export_csv ¶
Write the lab collection template as CSV.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str or Path
|
Output file path. |
required |
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Path
|
Written file path. |
export_excel ¶
Write the lab collection template as Excel (requires doekit[export]).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str or Path
|
Output file path. |
required |
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Path
|
Written file path. |
from_design
classmethod
¶
Wrap an existing :class:Design.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Design to manage. |
required |
model
|
Model
|
Model specification; taken from |
None
|
responses
|
sequence of str
|
Response column names (default |
None
|
Returns:
| Type | Description |
|---|---|
Experiment
|
|
Examples:
from_dict
classmethod
¶
Rebuild an :class:Experiment from :meth:to_dict output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d
|
mapping
|
Serialized experiment snapshot. |
required |
Returns:
| Type | Description |
|---|---|
Experiment
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
For unsupported schema or missing design. |
from_goal
classmethod
¶
Build an experiment from :func:~doekit.recommend_design.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
goal
|
str
|
|
required |
factors
|
Factor count or explicit factor specification. |
required | |
budget
|
int
|
Run budget passed to the advisor. |
None
|
model_order
|
str
|
Model order for the advisor. |
None
|
responses
|
sequence of str
|
Response column names (default |
None
|
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Experiment
|
Initialized with design, model, and recommendation metadata. |
Examples:
ingest ¶
Attach measured responses and optionally fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
response
|
array-like, Mapping, or DataFrame
|
Measured responses (1d, 2d, dict, or DataFrame). |
required |
fit
|
bool
|
When |
True
|
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Experiment
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If response row count does not match |
Examples:
load
classmethod
¶
Load an experiment from a wave directory or snapshot file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str or Path
|
Wave directory, project path, or |
required |
Returns:
| Type | Description |
|---|---|
Experiment
|
|
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
When no experiment snapshot is found at |
multi_response_summary ¶
Summarize per-response fit quality and overall desirability.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
goals
|
mapping
|
Passed to :meth: |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Raises:
| Type | Description |
|---|---|
ValueError
|
If :meth: |
Examples:
next ¶
Propose the next batch of runs.
intent="learn" augments for information using the primary response;
intent="optimize" fits a surrogate and proposes runs that move the
result (multi-objective when multiple responses were ingested).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_add
|
int
|
Number of new runs to propose. |
4
|
intent
|
('learn', 'optimize')
|
Information vs optimization intent. |
"learn"
|
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
NextRunsProposal
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If :meth: |
Examples:
report ¶
Generate an HTML report via the presentation layer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**kwargs
|
Forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Path or object
|
Report artifact from the presentation layer. |
save ¶
Persist the experiment into a workspace wave or project.
target may be a :class:~doekit.presentation.workspace.Wave, an
:class:~doekit.presentation.workspace.ExperimentProject (creates a new
wave), or a filesystem path to either.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target
|
Wave, ExperimentProject, str, or Path
|
Persistence destination. |
required |
thresholds
|
mapping
|
Quality thresholds for the wave manifest. |
None
|
seed
|
int
|
RNG seed stored in wave metadata. |
None
|
write_report
|
bool
|
Generate an HTML report during sync. |
False
|
comparison
|
Optional artifacts to attach to the wave. |
None
|
|
next_runs
|
Optional artifacts to attach to the wave. |
None
|
Returns:
| Type | Description |
|---|---|
Wave
|
Synced wave (or new wave when |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
When |
to_dict ¶
Serialize to a JSON-safe dict (schema: doekit.Experiment/1).
Returns:
| Type | Description |
|---|---|
dict
|
Snapshot including design, model, responses, evaluation, and fits. |
desirability_scores ¶
Compute per-run Derringer-style desirability across responses.
For each response column, values are scaled to [0, 1] within the
observed range (maximize) or inverted (minimize). Overall desirability is
the geometric mean across responses.
Formulas
- Per column
j:d_j = (x - min) / (max - min)(or1 - d_jfor min goals). - Overall:
D = exp(mean(log(d_j)))(geometric mean).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
frame
|
DataFrame
|
Multi-response matrix (n_runs x k). |
required |
goals
|
mapping
|
|
None
|
Returns:
| Type | Description |
|---|---|
Series
|
Overall desirability per run (name |
Examples:
experiment ¶
experiment(
goal="screening",
factors=None,
budget=None,
design=None,
model=None,
responses=None,
**kwargs,
)
Factory: ed.experiment(...) returns an :class:Experiment.
Pass design= to wrap an existing design, or goal + factors to
run the advisor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
goal
|
str
|
Advisor goal when building from factors. |
"screening"
|
factors
|
Factor count or specification (required unless |
None
|
|
budget
|
int
|
Run budget for the advisor. |
None
|
design
|
Design
|
Existing design to wrap. |
None
|
model
|
Model
|
Model specification. |
None
|
responses
|
sequence of str
|
Response column names. |
None
|
**kwargs
|
Forwarded to :meth: |
{}
|
Returns:
| Type | Description |
|---|---|
Experiment
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If neither |
Examples:
Sequential DoE¶
sequential ¶
Sequential / adaptive DoE orchestration.
DesignComparison
dataclass
¶
Side-by-side quality diff of two designs (compare_designs).
Attributes:
| Name | Type | Description |
|---|---|---|
a_label, b_label |
str
|
Labels for the two designs. |
a, b |
dict
|
Efficiency dicts (and mean power) for each design. |
delta |
dict
|
|
worth_it |
bool or None
|
Heuristic: True if B gains precision/prediction enough to justify extra runs. |
summary |
str
|
One-line human verdict. |
table |
DataFrame
|
Metric comparison table. |
Examples:
>>> import doekit as ed
>>> a, b = ed.plackett_burman(6), ed.augment_design(ed.plackett_burman(6), n_add=2, seed=0)
>>> cmp = ed.compare_designs(a, b, seed=0)
>>> "D_efficiency" in cmp.delta
True
NextRunsProposal
dataclass
¶
NextRunsProposal(
added,
combined,
comparison,
criterion,
rationale,
caveats=list(),
active_terms=list(),
sigma_hat=None,
intent="learn",
acquisition=None,
best_so_far=None,
predicted_improvement=None,
pareto_front=None,
explore_exploit=None,
surrogate=None,
acquisition_values=None,
)
Result of :func:propose_next_runs.
Attributes:
| Name | Type | Description |
|---|---|---|
added |
Design
|
Design containing only the proposed new runs. |
combined |
Design
|
Original design with the new runs appended. |
comparison |
DesignComparison
|
Metric delta current vs combined. |
criterion |
str
|
Criterion used for augmentation (learn intent). |
rationale |
str
|
Human-readable justification. |
caveats |
list of str
|
Assumptions and limitations. |
active_terms |
list of str
|
Terms flagged as active when |
sigma_hat |
float or None
|
Residual sigma from the fit when |
intent |
str
|
|
acquisition |
(str, optional)
|
Acquisition function name (optimize intent). |
best_so_far |
(object, optional)
|
Best observed objective value(s). |
predicted_improvement |
(float, optional)
|
Top acquisition score for the first selected candidate. |
pareto_front |
(list, optional)
|
Pareto front snapshot (multi-objective optimize). |
explore_exploit |
(dict, optional)
|
Explore/exploit stance for optimize intent. |
surrogate |
(Surrogate, optional)
|
Fitted surrogate (optimize intent only). |
acquisition_values |
(ndarray, optional)
|
Acquisition scores over the candidate pool (optimize intent). |
Examples:
>>> import doekit as ed
>>> import numpy as np
>>> pb = ed.plackett_burman(6)
>>> y = np.random.default_rng(0).normal(size=pb.n_runs)
>>> prop = ed.propose_next_runs(pb, response=y, n_add=2, seed=0)
>>> prop.added.n_runs == 2
True
augment_design ¶
augment_design(
design,
n_add,
model=None,
criterion="D",
candidates=None,
n_candidates=200,
n_starts=5,
seed=None,
)
Augment an existing design with n_add D/I-optimal new runs.
The current runs are fixed; new points are chosen from a candidate set to maximize the chosen criterion on the combined design.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Already-executed (or planned) design to keep. |
required |
n_add
|
int
|
Number of new runs to append. |
required |
model
|
Model
|
Model for the information matrix; taken from |
None
|
criterion
|
('D', 'I', 'A', 'G', 'E', 'T')
|
Optimality criterion for the combined design. |
"D"
|
candidates
|
Design
|
Candidate set; a grid/random cover of the factor region is built if omitted. |
None
|
n_candidates
|
int
|
Size of the auto-generated candidate set when |
200
|
n_starts
|
int
|
Independent greedy starts (best is kept). |
5
|
seed
|
int
|
RNG seed. |
None
|
Returns:
| Type | Description |
|---|---|
Design
|
Combined design (original rows + new rows). Metadata includes
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Notes
Current runs are fixed; only new rows are optimized. Uses greedy growth plus exchange refinement over multiple random starts.
Examples:
compare_designs ¶
compare_designs(
a,
b,
model=None,
effect_size=1.0,
sigma=1.0,
alpha=0.05,
n_region=4000,
seed=None,
a_label="current",
b_label="proposed",
run_cost=1.0,
)
Compare two designs on efficiencies, SPV and mean power.
Answers "is it worth paying for the extra runs in B?" with a transparent
metric delta table (same language as :func:~doekit.assessment.evaluation.evaluate).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
a
|
Design
|
Designs to compare (typically current vs augmented). |
required |
b
|
Design
|
Designs to compare (typically current vs augmented). |
required |
model
|
Model
|
Shared model; resolved from |
None
|
effect_size
|
Power-analysis assumptions. |
1.0
|
|
sigma
|
Power-analysis assumptions. |
1.0
|
|
alpha
|
Power-analysis assumptions. |
1.0
|
|
n_region
|
int
|
Region sampling for G/I metrics. |
4000
|
seed
|
int
|
Region sampling for G/I metrics. |
4000
|
a_label
|
str
|
Display labels. |
'current'
|
b_label
|
str
|
Display labels. |
'current'
|
run_cost
|
float
|
Relative cost per extra run (used only in the heuristic verdict). |
1.0
|
Returns:
| Type | Description |
|---|---|
DesignComparison
|
Side-by-side metrics, deltas, and |
Notes
worth_it is True when D or G rises >= 5 pts, mean SPV drops >= 10%,
or mean power rises >= 0.05 — scaled by extra run cost.
Examples:
propose_next_runs ¶
propose_next_runs(
design,
response=None,
n_add=4,
model=None,
criterion="D",
candidates=None,
budget=None,
priorities=None,
effect_size=1.0,
sigma=1.0,
alpha=0.05,
n_region=4000,
n_candidates=200,
n_starts=5,
seed=None,
active_p=0.05,
*,
intent="learn",
objectives=None,
goals=None,
goal="max",
acquisition=None,
surrogate="auto",
kappa=2.0,
xi=0.01,
)
Propose the next batch of runs for a sequential experiment.
Two intents share one entry point:
intent="learn"(default) — classical D/I-optimal augmentation: runs that sharpen the model (unchanged behavior). Withoutresponseit augments by information; withresponseit estimates residual sigma and flags active terms (p <active_p).intent="optimize"— surrogate + acquisition: fits a :class:~doekit.assessment.surrogate.Surrogate(GP with the OLS surface as prior mean, or plain OLS), then proposes runs that move the result toward the optimum. Supports multi-objective via Pareto/EHVI.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Current design (runs already done or committed). |
required |
response
|
array - like
|
Measured responses. For |
None
|
n_add
|
int
|
Number of new runs to propose. Capped by |
4
|
model
|
Model
|
Model for augmentation / the surrogate prior mean. |
None
|
criterion
|
str
|
Augmentation criterion (learn intent only). |
"D"
|
candidates
|
Design
|
Candidate set for new runs (both intents). |
None
|
budget
|
int
|
Maximum total runs (current + new). |
None
|
priorities
|
dict
|
Reserved for future ranking of criteria; accepted for API stability. |
None
|
effect_size
|
Power assumptions ( |
1.0
|
|
sigma
|
Power assumptions ( |
1.0
|
|
alpha
|
Power assumptions ( |
1.0
|
|
n_region
|
int
|
Evaluation / search controls. |
4000
|
n_candidates
|
int
|
Evaluation / search controls. |
4000
|
n_starts
|
int
|
Evaluation / search controls. |
4000
|
seed
|
int
|
Evaluation / search controls. |
4000
|
active_p
|
float
|
p-value cutoff for listing active terms when |
0.05
|
intent
|
('learn', 'optimize')
|
Whether to augment for information or optimize the response. |
"learn"
|
objectives
|
list of str
|
Column names when |
None
|
goals
|
dict
|
|
None
|
goal
|
('max', 'min')
|
Direction for a single objective when |
"max"
|
acquisition
|
str
|
Acquisition function: |
None
|
surrogate
|
('auto', 'ols', 'gp')
|
Surrogate backend (optimize intent). |
"auto"
|
kappa
|
float
|
UCB exploration weight. |
2.0
|
xi
|
float
|
EI/PI exploration margin. |
0.01
|
Returns:
| Type | Description |
|---|---|
NextRunsProposal
|
Proposed runs, combined design, comparison, and intent-specific fields. |
Raises:
| Type | Description |
|---|---|
ValueError
|
For unknown |
Notes
intent="learn" uses classical D/I-optimal augmentation; intent="optimize"
fits a surrogate and selects runs by acquisition (EI/UCB/PI/EHVI).
Examples:
BO bridge¶
bo ¶
Thin bridge from Bayesian-optimization search spaces to doekit candidates.
This module does not replace the transparent advisor. It only turns a
bound specification (or an optional scikit-optimize / Optuna space) into a
:class:~doekit.domain.design.Design candidate set that
:func:~doekit.orchestration.sequential.augment_design /
:func:~doekit.orchestration.sequential.propose_next_runs can consume — so BO
and classical DoE share efficiencies / FDS / power as a common language.
Optional extras: pip install "doekit[bo]" pulls scikit-optimize when you
want candidates_from_skopt_space.
candidates_from_bounds ¶
Sample a candidate :class:Design from factor bounds.
Builds a random candidate set over the declared factors for
:func:~doekit.orchestration.sequential.augment_design or optimal subsampling.
Attaches a main-effects model when none is provided.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bounds
|
sequence
|
Factors, or tuples |
required |
n
|
int
|
Number of candidate runs. |
200
|
model
|
Model
|
Attached to the returned design when provided. |
None
|
seed
|
int
|
RNG seed. |
None
|
Returns:
| Type | Description |
|---|---|
Design
|
Candidate set suitable for |
Examples:
candidates_from_skopt_space ¶
Build candidates from a skopt.space.Space (requires doekit[bo]).
Maps scikit-optimize dimensions to doekit factors and samples n points
with a fixed RNG seed for reproducibility.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
space
|
Space
|
Search space. |
required |
n
|
int
|
Number of candidate points (via dimension bounds). |
200
|
names
|
sequence of str
|
Dimension names; defaults to |
None
|
model
|
Model
|
Attached model. |
None
|
seed
|
int
|
RNG seed. |
None
|
Returns:
| Type | Description |
|---|---|
Design
|
Candidate design with |
Raises:
| Type | Description |
|---|---|
ImportError
|
If scikit-optimize is not installed. |
Report¶
report ¶
Presentation façade: narrative → render → write.
report_html ¶
report_html(
design,
response=None,
model=None,
output_dir="report",
filename=None,
title=None,
effect_size=1.0,
sigma=1.0,
alpha=0.05,
thresholds=None,
seed=None,
self_contained=False,
lang="en",
open_browser=False,
)
Generate an HTML report of the design (and its analysis if responses are given).
Without response it produces a design-quality report; with response,
a complete one (analysis + anomalies). Output is either a portable folder
(index.html, CSS, PNGs, CSVs) or a single self-contained HTML file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
The design (and, with |
required |
response
|
array - like
|
Measured response per run; enables the analysis and anomaly sections. |
None
|
model
|
Model
|
Model to evaluate/fit; resolved from the design if omitted. |
None
|
output_dir
|
str or Path
|
In folder mode, the report folder itself (holds |
"report"
|
filename
|
str
|
Name of the main HTML file (default |
None
|
title
|
str
|
Report title; a default is built from the design kind. |
None
|
effect_size
|
float
|
Passed to the evaluation (power analysis). |
1.0
|
sigma
|
float
|
Passed to the evaluation (power analysis). |
1.0
|
alpha
|
float
|
Passed to the evaluation (power analysis). |
1.0
|
thresholds
|
dict
|
Overrides for the quality thresholds ( |
None
|
seed
|
int
|
Seed for the region sampling of the evaluation. |
None
|
self_contained
|
bool
|
If |
False
|
lang
|
('en', 'es')
|
Report language. |
"en"
|
open_browser
|
bool
|
Open the written report in the web browser. |
False
|
Returns:
| Type | Description |
|---|---|
Path
|
Path to the written HTML file (the |
Raises:
| Type | Description |
|---|---|
ImportError
|
If matplotlib is not installed ( |
Notes
Narrative text is rule-based (deterministic, no LLM) for reproducibility.
Examples:
report_summary ¶
report_summary(
design,
response=None,
model=None,
effect_size=1.0,
sigma=1.0,
alpha=0.05,
seed=None,
thresholds=None,
lang="en",
blocks=None,
cov_type="nonrobust",
groups=None,
)
Return the experiment's semantic guide without writing HTML.
Same narrative content as the report (methodology, executive summary, recommendations, quality and anomalies) but as a data structure: useful to show inline in a notebook, or for an agent (MCP) to consume.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Executed (or planned) design. |
required |
response
|
array - like
|
Measured response; enables fit, ANOVA and anomaly sections. |
None
|
model
|
Model
|
Model for evaluation and fit; taken from the design if omitted. |
None
|
effect_size
|
float
|
Passed to :func: |
1.0
|
sigma
|
float
|
Passed to :func: |
1.0
|
alpha
|
float
|
Passed to :func: |
1.0
|
seed
|
int
|
Seed for region sampling in the evaluation. |
None
|
thresholds
|
dict
|
Overrides for quality thresholds ( |
None
|
lang
|
('en', 'es')
|
Language of the methodology/summary/recommendations prose. |
"en"
|
blocks
|
str or array - like
|
Fixed blocks for the OLS fit (see :func: |
None
|
cov_type
|
str
|
Covariance type for the OLS fit. |
"nonrobust"
|
groups
|
str or array - like
|
If set, also fit a mixed model and include |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Examples:
run_report_arg ¶
Normalize the functions' report= argument and generate the HTML.
report accepts None/False (do nothing), True (default folder
report/), a folder path (str/Path) or an options dict for
:func:report_html. Returns the written Path or None.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Design to report on. |
required |
response
|
array - like
|
Measured response per run. |
None
|
model
|
Model
|
Model for evaluation and fit. |
None
|
report
|
(None, bool, str, Path or dict)
|
Report trigger and options (see body). |
None
|
**extra
|
Additional keyword arguments forwarded to :func: |
{}
|
Returns:
| Type | Description |
|---|---|
Path or None
|
Path to the written HTML when a report was generated. |
Examples:
Export¶
export ¶
Export run sheets and lab collection templates (CSV / Excel).
Run sheets combine the coded design matrix with run_id and empty response
columns for bench data entry. Used by wave sync and the CLI --export flag.
export_csv ¶
Write a CSV run sheet to disk.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Design to export. |
required |
path
|
str or Path
|
Output CSV path (parent directories are created). |
required |
response_names
|
sequence of str
|
Response column names passed to :func: |
None
|
**kwargs
|
Forwarded to :meth: |
{}
|
Returns:
| Type | Description |
|---|---|
Path
|
Resolved path of the written file. |
export_excel ¶
Write an Excel run sheet (requires openpyxl / doekit[export]).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Design to export. |
required |
path
|
str or Path
|
Output |
required |
response_names
|
sequence of str
|
Response column names passed to :func: |
None
|
sheet_name
|
str
|
Worksheet name in the workbook. |
``"runs"``
|
Returns:
| Type | Description |
|---|---|
Path
|
Resolved path of the written file. |
Raises:
| Type | Description |
|---|---|
ImportError
|
When |
run_sheet ¶
Build a lab collection template from a design matrix.
Copies the coded factor columns and appends empty response columns for
bench entry. run_id is 1-based row index for traceability.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
design
|
Design
|
Design whose factor matrix forms the template rows. |
required |
response_names
|
sequence of str
|
Response column names; defaults to |
None
|
include_run_id
|
bool
|
When True, insert a |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
Factor columns plus |
Examples: