Latent Space Definitions for LLMs, Generative Models, and Fine-Tuning #

Latent space #

A latent space, also called an embedding space, is an abstract high-dimensional representation where a model maps similar items to nearby vectors. It is usually learned automatically from data: latent variables capture hidden features and arrange items across a manifold that is often smaller and easier to compute over than the raw feature space. In LLMs, tokens and hidden states live in latent space, where semantic relationships between words, phrases, prompts, and responses are encoded before being decoded back into language.

Example: A model can place "cat" and "dog" near each other in latent space because both share semantic features, while placing "invoice" farther away.

Dictionary: https://dictionary.platphormnews.com/en/define/latent-space

Embedding space #

An embedding space is a latent vector space where items such as words, documents, images, users, or products are represented as numerical coordinates. Items with related meanings or features are positioned near one another, which lets models compare similarity, retrieve neighbors, cluster concepts, and perform operations such as interpolation or vector addition.

Example: A search system can embed both a question and an article into the same embedding space, then retrieve the article whose vector is closest to the question.

Dictionary: https://dictionary.platphormnews.com/en/define/embedding-space

Latent reasoning #

Latent reasoning is the idea that a language model can carry part of its reasoning process inside continuous hidden-state vectors instead of only through explicit words. In an LLM, the prompt is projected into high-dimensional representations, transformed through model layers, and decoded into text. Research on latent reasoning treats a model's final hidden state as a reusable representation of an intermediate thought, allowing reasoning to continue directly in latent space.

Example: Instead of forcing every reasoning step into written text, an experiment may feed a hidden-state vector back into the model and let the next step happen in latent space.

Dictionary: https://dictionary.platphormnews.com/en/define/latent-reasoning

Latent operations #

Latent operations are edits, traversals, or measurements performed on vectors inside a model's latent space. Common operations include shifting a vector along a learned direction, interpolating between two vectors, subtracting one concept vector from another, masking dimensions, or sampling nearby points. These operations are used to explore how a model organizes concepts and to guide generation without rewriting the whole model.

Example: A designer might interpolate between two image embeddings to create a smooth transition from one visual concept to another.

Dictionary: https://dictionary.platphormnews.com/en/define/latent-operations

Latent space surgery #

Latent space surgery is a model-editing technique that identifies directions in a latent space corresponding to concepts, styles, or behaviors, then adds, subtracts, or dampens those directions to change outputs or internal representations. Instead of retraining an entire model, practitioners can use targeted vector edits to nudge behavior, such as moving a music embedding from a classical direction toward a jazz direction.

Example: If a vector direction reliably represents sentiment, latent space surgery can increase or reduce that direction to alter the tone of generated text.

Dictionary: https://dictionary.platphormnews.com/en/define/latent-space-surgery

Latent-space fine-tuning #

Latent-space fine-tuning describes how adaptation methods such as LoRA, prefix tuning, and adapters change a pretrained model by learning small parameter sets that shift, rotate, or redirect internal representations. Most of the base model stays frozen while the added parameters steer latent vectors toward a new domain, vocabulary, tone, or task. Cloud tools such as AWS SageMaker and Amazon Bedrock can support this workflow by training adapters and exposing embeddings for inspection.

Example: A legal-domain LoRA can teach an off-the-shelf LLM to arrange legal jargon more usefully in its latent space without fully retraining the model.

Dictionary: https://dictionary.platphormnews.com/en/define/latent-space-fine-tuning