Explore the details behind the poster.
Low-cost pseudo-labels guide a conditional diffusion model toward high-performing regions using only a limited number of real function evaluations.
The objective can be queried, but its formula and gradients are unavailable.
We seek a global maximizer \(x^*\in\arg\max_{x\in\mathcal X} H(x)\), where \(H\) is only accessible through pointwise evaluations.
An evaluation may involve an expensive simulation or physical experiment. In other settings, it may require a full model-training run. Exhaustive search is therefore impractical.
This challenge becomes more pronounced in high-dimensional search spaces, where maintaining adequate coverage requires increasingly many evaluations.
Exploration searches broadly to discover promising regions.
Exploitation concentrates evaluations around the best solutions found so far.
GBO-CD balances these objectives by learning a sampling distribution that progressively concentrates on high-performing regions.
A quick glossary for the main ideas used in the poster.
Generative optimization learns a probability distribution over candidate solutions instead of improving only one point. Sampling from this distribution creates many possible solutions and helps search complex spaces.
Pseudo-samples are inexpensive generated candidates that have not been evaluated by the true objective function. They form a large population that represents the current search distribution.
A pseudo-label is an estimated objective value assigned without evaluating the true objective. In GBO-CD, each pseudo-sample receives the value of its nearest evaluated point.
The real-evaluation budget is the number of times an algorithm may call the true objective function. It is the main cost used to compare methods fairly.
Exact evaluations provide reliable scores; pseudo-labels and the conditional diffusion model expand the search at low cost.
How pseudo-labels support Gibbs sampling through the conditional diffusion model.
The evaluated dataset \(D_k=\{(x_i,H(x_i))\}_{i=1}^{N_k}\) contains scarce but reliable objective information. The much larger pseudo-sample set \(Z_k=\{z_i\}_{i=1}^{M_k}\) provides a particle representation of the evolving search distribution.
For each \(z\in Z_k\), GBO-CD transfers the value of the nearest evaluated point:
This separation lets the method perform most distributional updates without paying for thousands of additional objective evaluations.
The objective value \(H(x)\) is converted into a sampling weight. Candidates with higher values receive exponentially more probability:
The temperature \(T\) controls concentration. High \(T\) makes the distribution flatter and encourages exploration across many regions. As \(T\) decreases, probability shifts toward the highest-value regions, increasing exploitation.
Optimization therefore becomes a sampling process: gradually cool the distribution until most samples appear near the global maximizer.
Instead of rebuilding the search distribution from scratch, GBO-CD updates the previous distribution using the change in inverse temperature:
After normalization, these weights describe how probability should shift as the system cools. This is the condition learned by the conditional diffusion model.
Within the conditional diffusion model, the forward diffusion process gradually adds noise to data. The reverse process learns to remove this noise step by step, ultimately transforming random noise into new candidates.
The conditional diffusion model receives the incremental Gibbs label as extra information while denoising. It learns how candidate locations relate to changes in Gibbs importance. This guidance enables the model to generate samples that track the next target distribution.
The temperature decreases across optimization iterations:
For \(\Delta H=H(x_a)-H(x_b)>0\), the relative Boltzmann weight is
Samples remain spread across the search space.
Sampling concentrates in the best regions found so far.
Each function makes optimization difficult in a different way.
A nearly flat outer region and many local maxima make premature convergence easy.
Search domain: \([-32.768,32.768]^d\)
Global maximum: \(f(\mathbf{x}^*)=0\)
At: \(\mathbf{x}^*=(0,\ldots,0)\)
Repeated oscillations and non-smooth absolute-value ridges create many local maxima across the search space.
Search domain: \([-10,10]^d\)
Global maximum: \(f(\mathbf{x}^*)=0\)
At: \(\mathbf{x}^*=(0,\ldots,0)\)
A rugged landscape with narrow valleys tests whether the search can reach a small optimal region.
Search domain: \([-10,10]^d\)
Global maximum: \(f(\mathbf{x}^*)=0\)
At: \(\mathbf{x}^*=(1,\ldots,1)\)
Each coordinate depends on its neighbors, so the dimensions cannot be optimized independently.
Boundary convention: \(x_0=x_d,\;x_{d+1}=x_1\)
Search domain: \([-10,10]^d\)
Global maximum: \(f(\mathbf{x}^*)=0\)
At: \(\mathbf{x}^*=(0,\ldots,0)\)
The experiments maximize the negative form of each standard benchmark, so higher objective values are better and the known optimum is 0.
They represent two established ways to search without gradients.
CE repeatedly fits a sampling distribution to the best candidates found so far.
Why it matters:It provides a distribution-learning baseline without a conditional diffusion model.
PSA uses a population and a cooling temperature to gradually shift from exploration to exploitation.
Why it matters:It provides a classical temperature-driven search baseline.
Together, CE and PSA test whether the conditional diffusion model adds value beyond distribution fitting and classical annealing.
The comparison uses matched initialization and equal real-evaluation budgets.
We first sample 50 points, \(X_0\), uniformly from the full search domain using a fixed random seed. The real objective function is evaluated once at these points to obtain \(y_0\).
The same immutable pair \((X_0,y_0)\) is used by GBO-CD and both baselines. Therefore, all three methods start from exactly the same locations and objective values.
After initialization, every method evaluates 50 new candidates per iteration. This gives the same real-evaluation budget: \(50(T+1)\), including the initial 50 evaluations. Low-cost pseudo-labels generated by GBO-CD do not count as additional real function evaluations.
Why the approximate search distribution should improve as the algorithm gathers information.
As more exact evaluations are added, the evaluated set covers the domain more densely. Under regularity assumptions such as a Lipschitz objective, a nearby evaluated point has a similar objective value, so nearest-neighbor pseudo-labels become more accurate.
More accurate pseudo-labels make the estimated Gibbs density closer to the ideal density defined by the unknown true objective. The theory separates this pseudo-objective error from generative distribution-tracking error.
As \(T_k\) decreases, the ideal Gibbs distribution assigns exponentially higher probabilities to points with higher objective values. With sufficiently gradual cooling, exploration is retained long enough for the evaluated set to improve.
If the learned sampling distribution tracks the pseudo-Gibbs sequence and the pseudo-Gibbs sequence tracks the ideal one, generated candidates increasingly concentrate near a global maximizer.
The distribution-level proof begins with objective approximation. It then establishes Gibbs-distribution tracking before showing concentration near the optimum.
The approach reduces expensive objective calls, but it does not remove every computational or modeling challenge.
A conditional diffusion model is retrained or updated across optimization iterations. Objective evaluations may be scarce, but neural-network training can still require substantial time and computational resources.
Nearest neighbors may be far apart in high dimensions. When local coverage is weak, a transferred objective value may be an inaccurate label for a pseudo-sample.
The current theoretical picture relies on regularity and distribution-tracking assumptions. Practical conditional diffusion model approximation error may not be negligible.
Important extensions include faster training and stronger pseudo-label estimators. Further work should address stochastic objectives while testing a wider range of high-dimensional problems.
This page provides intuitive explanations and supplementary details to help readers better understand the ideas presented in the poster.