Generative Model Families — LLMs, Diffusion, GANs, VAEs and Multimodal

What "generative" means, and why there are several families

A generative model learns the statistical shape of a body of data (text, images, audio, table rows) and can then sample new instances that look like it came from the same source. A discriminative model answers "which class is this?"; a generative model answers "what would a plausible example look like?". The families below differ in how they represent that distribution and how they sample from it.

Neural network basics in one paragraph

All five families are built from neural networks: stacks of layers, each layer multiplying its input by a matrix of numbers (the parameters or weights), adding a bias and applying a non-linear function. Training means showing the network many examples, measuring an error with a loss function, and adjusting every parameter slightly in the direction that lowers the loss; that direction is computed by backpropagation and applied by gradient descent.

Why it matters

• Consultants are asked "which AI?" before "which SAP product?"; a clear map of the five families stops the default answer "an LLM" being applied to tables, images and documents alike.
• Each family fails differently (hallucination, mode collapse, blurry samples, slow sampling), so the failure mode drives the controls you put around it, such as grounding, content filtering and human review.
• The metrics differ by family (perplexity, FID, human review), so a vendor claim of "better" is only meaningful if you know which yardstick it uses.

Key points

• Generative = learn a data distribution, then sample new instances from it.
• Autoregressive LLMs predict the next token; the Transformer (arXiv:1706.03762) made this scale.
• Diffusion = learned denoising of noise into an image; latent diffusion (arXiv:2112.10752) works in a compressed space and takes text conditioning.
• GANs pit a generator against a discriminator; they are fast but unstable and prone to mode collapse.
• VAEs encode to a latent space and decode back; they live on as the autoencoder inside latent diffusion.
• Multimodal models share a representation across media (CLIP, arXiv:2103.00020, 400 million image-text pairs).
• Evaluation: perplexity for language models, FID and Inception Score for images, human evaluation for open-ended output.
• In SAP you mostly meet LLMs (generative AI hub, Joule), tabular models (SAP-RPT-1) and document extraction (SAP Document AI); image and video generation is largely outside the stack.

Terms used on this page

Generative model

Model that learns a data distribution and samples new instances.

Token

Unit of text a language model reads and predicts.

Transformer

Attention-based network architecture introduced in arXiv:1706.03762.

Diffusion model

Model trained to remove noise step by step, then used to generate from pure noise.

Latent space

Compressed representation where a VAE or latent diffusion model works.

Mode collapse

GAN failure where the generator covers only a few output types.

Reparameterisation trick

Rewrites a random sample as a deterministic function plus noise so gradients can flow.

Perplexity

Exponentiated average negative log-likelihood per token; lower is better, tokeniser-dependent.

Sources

1. arXiv:1706.03762 — Attention Is All You Need (submitted 12 June 2017; accessed 18 Oct 2026)
2. arXiv:1406.2661 — Generative Adversarial Networks (submitted 10 June 2014; accessed 18 Oct 2026)
3. arXiv:1312.6114 — Auto-Encoding Variational Bayes (submitted 20 December 2013; accessed 18 Oct 2026)
4. arXiv:2006.11239 — Denoising Diffusion Probabilistic Models (June 2020; accessed 18 Oct 2026)
5. arXiv:2112.10752 — High-Resolution Image Synthesis with Latent Diffusion Models (December 2021; accessed 18 Oct 2026)
6. arXiv:2103.00020 — Learning Transferable Visual Models From Natural Language Supervision (February 2021; accessed 18 Oct 2026)
7. arXiv:1706.08500 — GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (introduces FID; June 2017; accessed 18 Oct 2026)
8. Hugging Face Transformers documentation — Perplexity of fixed-length models (accessed 18 Oct 2026)
9. SAP Learning — Discovering the Orchestration Service, generative AI hub course (modules: grounding, templating, content filtering, data masking, translation; accessed 18 Oct 2026)
10. SAP Architecture Center — Document Processing with SAP Document AI (page last updated 3 June 2026; 85-95% accuracy figure is vendor-reported; accessed 18 Oct 2026)
11. SAP Developer tutorial — extract data with generative AI in SAP Document AI (field descriptions act as LLM prompts; accessed 18 Oct 2026 via search result)
12. Towards Data Science — One Model to Rule Them All: SAP-RPT-1 and the Future of Tabular Foundation Models (secondary source, cited via search summary; accessed 18 Oct 2026)
13. ERP Today — SAP-RPT-1 Brings Tabular Foundation Models to Enterprise AI (secondary source, cited via search summary; accessed 18 Oct 2026)
14. SAP product page — SAP-RPT (not retrievable on 18 Oct 2026, HTTP 403; verify directly)