0
# Web User Interface
1

2
Streamlit-based web interface for interactive model testing and development. The web UI provides a browser-based chat interface with configurable parameters, real-time model interaction, and an intuitive graphical interface for users who prefer web-based interaction over command-line tools.
3

4
## Capabilities
5

6
### Web UI Launch
7

8
Launch the Streamlit web interface for interactive model testing and configuration.
9

10
```python { .api }
11
def webui() -> None:
12
    """
13
    Launch the Streamlit web user interface.
14
    
15
    Creates a browser-based interface for interacting with PyLLaMACpp models,
16
    providing chat functionality, parameter configuration, and real-time
17
    text generation monitoring.
18
    """
19

20
def run():
21
    """
22
    Run the web UI application.
23
    
24
    Alternative entry point for launching the web interface,
25
    typically used for programmatic access or custom deployments.
26
    """
27
```
28

29
### Basic Usage
30

31
Launch the web interface directly from Python:
32

33
```python
34
from pyllamacpp.webui import webui
35

36
# Launch the web interface
37
webui()
38
```
39

40
Or use the run function:
41

42
```python
43
from pyllamacpp.webui import run
44

45
# Alternative launch method
46
run()
47
```
48

49
The web interface will start a local Streamlit server accessible through your browser, typically at `http://localhost:8501`.
50

51
### Web UI Features
52

53
The web interface provides:
54

55
1. **Model Selection**: Browse and select GGML model files
56
2. **Parameter Configuration**: Adjust generation parameters with sliders and inputs
57
3. **Interactive Chat**: Real-time chat interface with the loaded model
58
4. **Response Streaming**: Live display of generated tokens as they are produced
59
5. **Session History**: Maintain conversation history within the session
60
6. **Parameter Presets**: Save and load commonly used parameter configurations
61
7. **Model Information**: Display model metadata and performance statistics
62

63
### Interface Components
64

65
The web UI includes several key components:
66

67
#### Model Configuration Panel
68
- **Model Path Selection**: File browser for selecting GGML model files
69
- **Context Parameters**: Configure n_ctx, seed, f16_kv, and other model settings
70
- **GPU Settings**: Configure GPU layer offloading and memory options
71

72
#### Generation Parameters Panel
73
- **Sampling Controls**: Temperature, top-p, top-k sliders
74
- **Length Controls**: n_predict, n_keep configuration
75
- **Advanced Parameters**: Mirostat, repetition penalty, frequency penalty
76
- **Performance Settings**: Thread count, batch size optimization
77

78
#### Chat Interface
79
- **Message Input**: Text area for entering prompts and questions
80
- **Response Display**: Real-time streaming output with syntax highlighting  
81
- **Conversation History**: Scrollable chat history with user/AI distinction
82
- **Action Buttons**: Send, clear, reset context controls
83

84
### Configuration Schema Integration
85

86
The web UI uses the same parameter schemas as the CLI for consistency:
87

88
```python
89
# Integration with CLI parameter schemas
90
from pyllamacpp.constants import GPT_PARAMS_SCHEMA
91

92
# Dynamic UI generation based on parameter schema
93
def _generate_config_ui(model_name, config_schema):
94
    """Generate UI components based on parameter schema."""
95
    
96
def _config_ui(config_name: str, key: str, config: dict):
97
    """Generate individual parameter UI component."""
98
    
99
def _get_config_from_session_state(model_name: str, config_schema: dict) -> dict:
100
    """Extract configuration from Streamlit session state."""
101
```
102

103
### Advanced Usage
104

105
#### Custom Web UI Deployment
106

107
Deploy the web UI with custom configuration:
108

109
```python
110
import streamlit as st
111
from pyllamacpp.webui import webui
112
from pyllamacpp.model import Model
113

114
# Custom Streamlit configuration
115
st.set_page_config(
116
    page_title="Custom PyLLaMACpp Interface",
117
    page_icon="🦙",
118
    layout="wide"
119
)
120

121
# Launch with custom settings
122
webui()
123
```
124

125
#### Programmatic Integration
126

127
Integrate web UI components into custom applications:
128

129
```python
130
import streamlit as st
131
from pyllamacpp.webui import _create_model, _get_config_from_session_state
132
from pyllamacpp.constants import GPT_PARAMS_SCHEMA
133

134
def custom_model_interface():
135
    st.title("Custom Model Interface")
136
    
137
    # Model selection
138
    model_path = st.file_uploader("Select GGML Model", type=['bin', 'ggml'])
139
    
140
    if model_path:
141
        # Create model with UI configuration
142
        model = _create_model(model_path.name, n_ctx=2048)
143
        
144
        # Get parameters from UI
145
        params = _get_config_from_session_state("custom_model", GPT_PARAMS_SCHEMA)
146
        
147
        # Chat interface
148
        if prompt := st.chat_input("Enter your message"):
149
            with st.chat_message("user"):
150
                st.write(prompt)
151
            
152
            with st.chat_message("assistant"):
153
                response_placeholder = st.empty()
154
                full_response = ""
155
                
156
                for token in model.generate(prompt, **params):
157
                    full_response += token
158
                    response_placeholder.write(full_response)
159

160
custom_model_interface()
161
```
162

163
### Session State Management
164

165
The web UI manages state across user interactions:
166

167
```python
168
# Session state initialization
169
def _init_session_state(key: str, val: any):
170
    """Initialize session state variable if not exists."""
171

172
# Key generation for unique UI components
173
def _get_key(model_name: str, config_name: str) -> str:
174
    """Generate unique key for UI components."""
175
    return f"{model_name}-{config_name}"
176
```
177

178
### Real-time Features
179

180
#### Live Token Streaming
181

182
The web UI displays tokens as they are generated:
183

184
```python
185
def new_text_generated(text):
186
    """Callback function for new text generation."""
187
    # Real-time display in Streamlit interface
188
    st.write(text, end="")
189
```
190

191
#### Performance Monitoring
192

193
Display real-time performance metrics:
194

195
- **Tokens per second**: Generation speed monitoring
196
- **Memory usage**: RAM and GPU memory consumption
197
- **Model load time**: Initial model loading performance
198
- **Response latency**: Time from prompt to first token
199

200
### Web UI Configuration
201

202
#### Streamlit Configuration
203

204
The web UI can be configured through Streamlit's config system:
205

206
```toml
207
# .streamlit/config.toml
208
[server]
209
port = 8501
210
address = "0.0.0.0"
211

212
[theme]
213
primaryColor = "#FF6B6B"
214
backgroundColor = "#FFFFFF"
215
secondaryBackgroundColor = "#F0F2F6"
216
textColor = "#262730"
217

218
[browser]
219
gatherUsageStats = false
220
```
221

222
#### Custom Themes
223

224
Apply custom styling to the web interface:
225

226
```python
227
import streamlit as st
228

229
# Custom CSS styling
230
st.markdown("""
231
<style>
232
.stChatMessage {
233
    padding: 1rem;
234
    border-radius: 0.5rem;
235
    margin-bottom: 1rem;
236
}
237

238
.user-message {
239
    background-color: #E3F2FD;
240
    border-left: 4px solid #2196F3;
241
}
242

243
.assistant-message {
244
    background-color: #F3E5F5;
245
    border-left: 4px solid #9C27B0;
246
}
247
</style>
248
""", unsafe_allow_html=True)
249
```
250

251
### Deployment Options
252

253
#### Local Development
254

255
Run the web UI for local development:
256

257
```bash
258
# Direct Python execution
259
python -c "from pyllamacpp.webui import webui; webui()"
260

261
# Or using Streamlit directly
262
streamlit run -m pyllamacpp.webui
263
```
264

265
#### Production Deployment
266

267
Deploy the web UI in production environments:
268

269
```bash
270
# Docker deployment
271
docker run -p 8501:8501 -v /models:/models streamlit-pyllamacpp
272

273
# Cloud deployment with specific configuration
274
streamlit run --server.port 8080 --server.address 0.0.0.0 -m pyllamacpp.webui
275
```
276

277
### Security Considerations
278

279
When deploying the web UI:
280

281
1. **Model Access**: Restrict access to model files and directories
282
2. **Network Security**: Use HTTPS in production deployments
283
3. **Resource Limits**: Configure memory and CPU limits
284
4. **Authentication**: Implement user authentication for sensitive deployments
285
5. **Input Validation**: Sanitize user inputs and prompts
286

287
### Integration with External Tools
288

289
The web UI can be extended with additional functionality:
290

291
```python
292
# Integration with monitoring tools
293
import streamlit as st
294
import plotly.graph_objects as go
295

296
def add_performance_dashboard():
297
    """Add performance monitoring dashboard."""
298
    col1, col2 = st.columns(2)
299
    
300
    with col1:
301
        st.metric("Tokens/sec", "45.2", "2.1")
302
    
303
    with col2:
304
        st.metric("Memory Usage", "3.2 GB", "-0.1 GB")
305
    
306
    # Performance graph
307
    fig = go.Figure()
308
    fig.add_trace(go.Scatter(y=[40, 42, 45, 43, 47], name="Tokens/sec"))
309
    st.plotly_chart(fig)
310
```
311

312
### Dependencies
313

314
The web UI requires additional dependencies:
315

316
```python
317
# Required packages
318
import streamlit as st
319
from streamlit import runtime
320
from streamlit_ace import st_ace, KEYBINDINGS, LANGUAGES, THEMES
321
from streamlit.web import cli as stcli
322
```
323

324
Install web UI dependencies:
325

326
```bash
327
pip install streamlit streamlit-ace
328
```