Add LLada2 MoE

[kernelpool]

This is a text diffusion model, so this support is experimental. However, text generation works just fine and you can adjust the steps, threshold, block length, etc.

mlx_lm.generate --model /Volumes/WD_EXTRA/models/catalyst/LLaDA2.0-mini-preview-4bit --prompt "what is the secure enclave processor?" --max-tokens 4096 --steps 32
==========
A **Secure Enclave Processor (SEP)** is a specialized hardware component designed to protect sensitive data and code execution in a device. It operates in a secure, sandboxed execution environment, separate from the main operating system and user applications. This isolation helps prevent unauthorized access or attacks, even if the main system is compromised.

### Key Features and Benefits of a Secure Enclave Processor:

1. **Isolation**:
   - The Secure Enclave runs in its own environment, separate from the main OS and apps.
   - This reduces the attack surface by keeping sensitive operations (like encryption or biometric processing) away from general access.

2. **Hardware-Based Security**:
   - Uses dedicated processor features (e.g., ARM’s TrustZone, Intel’s SGX, Apple’s Secure Enclave) to provide cryptographic operations and secure memory.
   - These features are often built into the CPU or processor architecture, offering strong protection.

3. **Secure Key Storage**:
   - Stores encryption keys and sensitive data in secure memory (e.g., Secure Enclave memory on Apple).
   - Prevents keys from being accessed by software on the device.

4. **Biometric Processing**:
   - Handles biometric data (e.g., fingerprint, facial recognition) for authentication.
   - Biometric data is typically encrypted or hashed and never leaves the device.

5. **PK protection**:
   - Protects private keys used in digital signatures (e.g., iPhone signing, app code signing).
   - Prevents attackers from stealing private keys.

6. **Tamper Detection**:
   - Many Secure Enclaves include built-in mechanisms to detect and resist physical or logical tampering.
   - If tampering occurs, the enclave can erase itself or trigger security response.

---

### Examples of Secure Enclave Processors:

- **Apple’s Secure Enclave (A7 and later)**:
  - Used in iOS devices.
  - Handles biometric authentication, key storage, and secure messaging.
  - Isolated by the **Apple TrustZone** technology.

- **ARM’s TrustZone (used in many ARM processors)**:
  - Provides hardware-level separation between privileged and non-privileged code.
  - Common in embedded systems and mobile devices.

- **Intel SGX (Software Guard Extensions)**:
  - Allows applications to create secure data enclaves within main memory.
  - Used in enterprise and high-security applications.

- **Qualcomm Secure Execution Environment (QSEE)**:
  - Used in Android devices.
  - Protects encryption, payments, and sensitive data.

---

### Use Cases:

- Biometric authentication (Face ID, Touch ID)
- Mobile payments (e.g., Apple Pay, Samsung Pay)
- Secure app data storage
- Device encryption and key management
- Secure communications (e.g., secure messaging)

---

### Summary:

A **Secure Enclave Processor** is a hardware-level security module that isolates sensitive operations and protects critical data on a device. It’s a fundamental component of modern secure computing, especially in mobile and embedded systems.<|role_end|><|endoftext|>
==========
Prompt: 26 tokens, 0.636 tokens-per-sec
Generation: 674 tokens, 16.498 tokens-per-sec
Peak memory: 9.934 GB

[kernelpool]

I realized there have been some past efforts at integrating diffusion models, so I'll leave this like this for now. It was a fun exercise nonetheless :) Here are generations from the supported models:

mlx-community/LLaDA2.0-mini-preview-4bit

mlx_lm.generate --model mlx-community/LLaDA2.0-mini-preview-4bit -p "who is albert einstein" -m 1024
==========
Albert Einstein (1879–1955) was a German-born theoretical physicist who is widely regarded as one of the most influential scientists in history. He is best known for his contributions to physics, particularly the development of **the theory of relativity**, which revolutionized our understanding of space, time, and gravity.

### Key Contributions:

1. **Special Theory of Relativity (1905):**
   - Introduced the idea that the laws of physics are the same for all non-accelerating observers.
   - Proposed that the speed of light in a vacuum is constant and independent of the motion of the source or observer.
   - Led to the concept of **time dilation** and **length contraction**.
   - Famous equation: **E = mc²**, which expresses the equivalence of mass and energy.

2. **General Theory of Relativity (1915):**
   - A comprehensive theory of gravity, describing it as the curvature of spacetime caused by mass and energy.
   - Predicted phenomena such as black holes, gravitational waves, and the bending of light by gravity.
   - Has been confirmed by observations and experiments, including the bending of starlight during a solar eclipse in 1919.

### Other Contributions:
- **Photoelectric Effect (1905):** Einstein explained how light can behave as particles (photons), leading to the development of **quantum theory**.
- **Brownian Motion:** Einstein provided a theoretical explanation for the random movement of particles suspended in a fluid, which helped confirm the existence of atoms and molecules.
- **Statistical Mechanics:** Made foundational contributions to the understanding of heat and molecular theory.

### Legacy:
- Einstein received the **Nobel Prize in Physics in 1921** for his explanation of the photoelectric effect, not for relativity.
- He became a global icon of science and intellect, known for his radical ideas and pacifist views.
- His work continues to influence modern physics, technology, and cosmology.

Einstein remains a symbol of human curiosity and the power of theoretical physics.
==========
Prompt: 24 tokens, 260.105 tokens-per-sec
Generation: 446 tokens, 19.671 tokens-per-sec
Peak memory: 9.832 GB

mlx-community/LLaDA2.0-flash-preview-4bit

mlx_lm.generate --model mlx-community/LLaDA2.0-flash-preview-4bit -p "who is albert einstein" -m 1024
==========
Albert Einstein (1879–1955) was a German-born theoretical physicist, widely regarded as one of the most influential scientists of all time. He is best known for developing the **theory of relativity**, one of the two pillars of modern physics (alongside quantum mechanics).

### Key Contributions:
1. **Special Theory of Relativity (1905)**:
   - Introduced the idea that the laws of physics are the same for all non-accelerating observers.
   - Proposed that the speed of light in a vacuum is constant, regardless of the observer's motion.
   - Led to the famous equation: **E = mc²**, which expresses the equivalence of mass and energy.

2. **General Theory of Relativity (1915)**:
   - Expanded the theory to include gravity, describing it not as a force but as a curvature of spacetime caused by mass and energy.
   - Predicted phenomena such as gravitational lensing, black holes, and the expansion of the universe.

3. **Photoelectric Effect (1905)**:
   - Explained how light can behave as discrete packets of energy (photons), which helped lay the foundation for quantum theory.
   - This work earned him the **Nobel Prize in Physics in 1921**.

### Life and Legacy:
- Born in Ulm, Germany, Einstein studied in Switzerland and later worked as a patent clerk in Bern.
- He became a professor in Europe and later emigrated to the United States in 1933 due to the rise of the Nazi regime.
- He became a U.S. citizen and was affiliated with the Institute for Advanced Study in Princeton, New Jersey.
- Einstein was also known for his pacifism, civil rights activism, and intellectual curiosity.

### Cultural Impact:
- His face has become synonymous with "genius."
- His work revolutionized our understanding of space, time, gravity, and the universe.

Albert Einstein remains a symbol of scientific innovation and human curiosity.
==========
Prompt: 24 tokens, 55.006 tokens-per-sec
Generation: 435 tokens, 5.561 tokens-per-sec
Peak memory: 58.731 GB