Virtual assistant chatgpt

virtual-assistant-chatgpt/README.md

@@ -0,0 +1,166 @@
+# Weather Virtual Assistant Example
+
+## Overview
+
+The Virtual Assistant (VA) sample demonstrates how to use NVIDIA NeMo LLM along with Riva AI Services to build a simple but complete conversational AI application. It demonstrates receiving input via speech from the user, interpreting the query via an intention recognition and slot filling approach, leveraging the NeMo LLM to generate a natural sounding human-like response, and speaking this back to the user in a natural voice.
+
+## Prerequisites
+
+- This demo uses NVIDIA Riva to support Speech AI capabilities like Automatic Speech Recognition (ASR) and Text-to-Speech (TTS). To run NVIDIA Riva Speech AI services, please ensure you have the pre-requisites mentioned [here](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html#data-center).
+- For running this sample application, you'll need: 
+    - Access to the [OpenAI platform](https://platform.openai.com/). You will require your [OpenAI API key](https://platform.openai.com/account/api-keys) to access the service through the API in this sample application.
+    - A Linux x86_64 environment with [pip](https://pypi.org/project/pip/) and Python 3.8+ installed.
+    - The [weatherstack API access key](https://weatherstack.com/documentation). The VA uses weatherstack for weather fulfillment, that is when the weather intents are recognized, the real-time weather information is fetched from weatherstack. Sign up to the free tier of [weatherstack](https://weatherstack.com/), and get your API access key.
+    - A microphone and speaker (for example, a Logitech H390 USB Computer Headset) to communicate with the app.
+
+
+### Setup
+
+1. Clone the NVIDIA Riva Sample Apps repo.
+
+2. Enter the home directory of the Virtual Assistant ChatGPT sample app:
+```bash
+cd /path/to/sample-apps/virtual-assistant-chatgpt
+```
+
+3. Create and enable a Python [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment). For example:
+```
+python3 -m venv apps-env
+source apps-env/bin/activate
+```
+
+After activating, checking the Python version should reveal the one you created the environment with. For example:
+```
+python3 --version
+```
+*Python 3.8.10*
+
+
+4. Install the libraries necessary for the virtual assistant, including the Riva and OpenAI client libraries:
+    1. Install weatherbot web application dependencies. `requirements.txt` captures all Python dependencies needed for weatherbot web application.
+        ```bash
+        pip install -r requirements.txt # Tested with Python 3.8
+        ```
+    2. Install the Riva client library.
+		```
+		pip install nvidia-riva-client
+		```
+	3. Install the OpenAI Client library
+        ```bash
+        pip install openai
+        ```
+
+### Running the demo
+1.  Start the Riva Speech Server, if not already done. Follow the steps in the [Riva Quick Start Guide](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html). This will allow Speech AI capabilities which are required for the demo. **Note the IP & port** where the Riva server is running. By default it will run at IP:50051.
+    1. In the `config.sh` script included in the 
+    [Riva Skills Quick Start](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/riva/resources/riva_quickstart) resource folder, set 
+    ```bash
+    service_enabled_asr=true
+    service_enabled_nlp=true
+    service_enabled_tts=true
+    service_enabled_nmt=false
+    ```
+    2. In `config.sh`, under 
+    ```bash
+    else
+      models_nlp=(
+      ...
+    ```
+    comment out or uncomment the names of the NLP models as desired. At a minimum, 
+    this sample app requires the BERT Base Intent Slot model. The BERT Base 
+    Punctuation and Capitalization model and BERT Base Named Entity Recognition 
+    model are also recommended. 
+
+    3. In a terminal, in the Riva Skills Quick Start resource folder's home 
+    directory, run 
+    ```bash
+    ./riva_init.sh
+    ```
+    to download and deploy the speech AI models. Then, run 
+    ```bash
+    ./riva_start.sh
+    ```
+    to start the Riva Server.
+
+2. Edit the configuration file [config.py](./config.py)
+    1. In `riva_config` set:
+        * The Riva speech server URL. This is the endpoint where the Riva services can be accessed.
+        * The [weatherstack API access key](https://weatherstack.com/documentation). The VA uses weatherstack for weather fulfillment, that is when the weather intents are recognized, real-time weather information is fetched from weatherstack. Sign up to the free tier of [weatherstack](https://weatherstack.com/), and get your API access key.
+    2. In `llm_config` set:
+        * The OpenAI API Access key
+        * (Optionally) you can also choose the GPT model to use. By default, 
+        this is set to "gpt-3.5-turbo", but check out 
+        https://platform.openai.com/docs/models for more options.
+
+The code snippets will look like the example below.
+```python3
+riva_config = {
+  "RIVA_SPEECH_API_URL": "<IP>:<PORT>", # Replace the IP & port with your hosted Riva endpoint
+   ...
+  "WEATHERSTACK_ACCESS_KEY": "<API_ACCESS_KEY>",  # Get your access key at - https://weatherstack.com/
+   ...
+}
+...
+llm_config = {
+    ...
+    "API_MODEL_NAME":"gpt-3.5-turbo",
+    "API_KEY": "<OPENAI_API_ACCESS_KEY>" # Get your access key at https://platform.openai.com/account/api-keys
+    ...
+}
+```
+
+3. Run the virtual assistant application
+```bash
+python3 main.py
+```
+
+4. Open the browser to **https://IP:8009/rivaWeather**, where the IP is for the machine where the application is running. For instance, go to <https://127.0.0.1:8009/rivaWeather/> if the app is running in your local machine.
+
+5. Speak to the virtual assistant through your microphone or type-in your text, asking a weather related query. To hear back text-to-speech audio of the LLM response, click on "Unmute System Speech" on the right bottom corner of the UI.
+
+`NOTE:` To learn about the call to the LLM Service, please refer to the `query_llm` method in `riva_local/chatbot/stateDM/Util.py`.
+
+## Sample Use Cases
+It is possible to ask the bot the following types of questions:
+
+* What is the weather in Berlin?
+
+* What is the weather?
+    * For which location?
+
+* What’s the weather like in San Francisco tomorrow?
+    * What about in California City?
+
+* What is the temperature in Paris on Friday?
+
+* How hot is it in Berlin today?
+
+* Is it currently cold in San Francisco?
+
+* Is it going to rain in Detroit tomorrow?
+
+* How much rain in Seattle?
+
+* Will it be sunny next week in Santa Clara?
+
+* Is it cloudy today?
+
+* Is it going to snow tomorrow in Milwaukee?
+
+* How much snow is there in Toronto currently?
+
+* How humid is it right now?
+
+* What is the humidity in Miami?
+
+* What's the humidity level in San Diego?
+
+## Limitations
+* The sample supports intents for weather, temperature, rain, humidity, sunny, cloudy and snowfall checks. It does not support general conversational queries or other domains.
+* The sample supports only 1 slot for city.
+* The sample supports up to four concurrent users. This restriction is because of the web framework (Flask and Flask-SocketIO) that is being used. The socket connection is to stream audio to (TTS) and from (ASR); you are unable to sustain more than four concurrent socket connections.
+* The chatbot application is not optimized for low latency in the case of multiple concurrent users.
+* Some erratic issues have been observed with the chatbot sample on the Firefox browser. The most common issue is the TTS output being taken in as input by ASR for certain microphone gain values.
+
+## License
+The [NVIDIA Riva License Agreement](https://developer.nvidia.com/riva/ga/license) is included with the product. Licenses are also available along with the model application zip file. By pulling and using the Riva SDK container, downloading models, or using the sample applications here, you accept the terms and conditions of these licenses. <br>

virtual-assistant-chatgpt/client/webapplication/cert.pem

@@ -0,0 +1,34 @@
+-----BEGIN CERTIFICATE-----
+MIIF8TCCA9mgAwIBAgIUCWhCFWAPSu7g+Tn0D4+oYfLn9xUwDQYJKoZIhvcNAQEL
+BQAwgYcxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRUwEwYDVQQH
+DAxTYW5GcmFuY2lzY28xEDAOBgNVBAoMB1FWQU5UVk0xCzAJBgNVBAsMAlFWMQsw
+CQYDVQQDDAJTRDEgMB4GCSqGSIb3DQEJARYRcXZhbnR2bUBnbWFpbC5jb20wHhcN
+MjAwMTE1MTg0NTAyWhcNMjEwMTE0MTg0NTAyWjCBhzELMAkGA1UEBhMCVVMxEzAR
+BgNVBAgMCkNhbGlmb3JuaWExFTATBgNVBAcMDFNhbkZyYW5jaXNjbzEQMA4GA1UE
+CgwHUVZBTlRWTTELMAkGA1UECwwCUVYxCzAJBgNVBAMMAlNEMSAwHgYJKoZIhvcN
+AQkBFhFxdmFudHZtQGdtYWlsLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCC
+AgoCggIBAM62M4FzhOfntQARneCixYTl2mi8owa2spm600Lopy8Fwg2e5jkY/pyg
+znK7foHBvW3eJKxENtwLKIUqNYyOA2F+96FW1fZ9MUF6zcYcccFimjjKr6tYrqDf
+GIqAgxOcv4Syv/DaIm3tKDwIYFFtB/kALyG0vBHWV40fvOoqTwCQUSbptM9GBRZe
++beMUS83Zwk4OPPIiSX+P+DEzsBAqUbqU50ilJrzM7VZ+QpVooF/De4moRDSknVs
+h/Zbn9a8Wbfb1XpBzm19mnO1vZ6CENUIFdCxvW4qrmfIu346bd+stXcwcJp7bLP0
+vmVnK5DpGIjmzo7n97nFJUO3kFykbJcWpTIwWSc7jsnak+HkwxXAYWZmz0sABDLA
+QBK37NuYCqsDxvOibC2X4Y0oRxYut8R0nUYXOtKHBvR5Ug9njIov9lsV8acIm7s9
+r0fignHgFOXmTDMIjK469LwXf0vd1Wy6tZxjNS2ZOG+eyc5sKHCyVP6dEl8xYH6i
+/oePn5PmzSr220comUny5NVkUewDYo63A90bQ4X3tdeI/XsPZDXhgQC36+wKFKgy
+6WULQhZT4EpxMbiYNF8IjTr6w/5mk1oOT4/nRedcb7/wxlZiL5r/DJlMVuP7zlfq
+n3CvLIlpiy6E2b1c7LU+mvVnkq0LQrD+yK30p3JSVwabm6GeY/tnAgMBAAGjUzBR
+MB0GA1UdDgQWBBRs9s5GEerZSVFVKGAUIOxNDbE7HDAfBgNVHSMEGDAWgBRs9s5G
+EerZSVFVKGAUIOxNDbE7HDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUA
+A4ICAQAv3d5cMLwb76ikYt37P10ggPOjyZwAs90mR/2f25+ADyUaX/wzHtitaA8t
+cXfpm4pDICRtJjg5hb6A2Wh+ws/gODbyD915sqYDC6pz0FTxh3n2/BcCQZJWa7Oy
+9k7RwixwIYjvGzkWaje9Xi5Jte7tBj4358QFUKKWiWarB+nl2MBAW3wKMRO5stst
+pNZ2keaMeG/a5/Ms0PLIeSxN5oUNazs1m3NLB7XFx1ewMKdouamhuCDRyoPixtfm
+pas94/IUdGyY6WYaZgGdQtBrM9ro8NtJNFiFQImI0NaQJQCgl45pmapceNzHDHgR
+XHnbirr7rRJX6UutrEHNCZcwA5sFQ++AtQycW20vTGJZ1cO+mCYvWEKN0PC4KRhY
+Aw8q8ROkX9RvxHl3WGFdpIFXAZtI8O1d7G9ySG1xBLWyYW7s58vdFnsqmjz4Y59b
+MWWCIBIAhxLFBRrZ7KEGUj0lJjEuRZ9lkfS4CAAX2gUSlgU/GfUWe8R3bzAqC57t
+awACY7PS3m29ILHuKEU4qRarSWdYaJP7ETc6TQDgxXVXA8NTQOMFj73zYILl7MYh
+DXvHzk8xKGWqUv2gmKylsqQZPm/mgn/dssOV19TB6LOszX2rJ0ddZ63on5wJqDGl
+6BiPGopQSm/mJPNfO3JjvetTEumfinvsac2eTZ99zbTiJ0Pyfw==
+-----END CERTIFICATE-----

virtual-assistant-chatgpt/client/webapplication/key.pem

@@ -0,0 +1,52 @@
+-----BEGIN PRIVATE KEY-----
+MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQDOtjOBc4Tn57UA
+EZ3gosWE5dpovKMGtrKZutNC6KcvBcINnuY5GP6coM5yu36Bwb1t3iSsRDbcCyiF
+KjWMjgNhfvehVtX2fTFBes3GHHHBYpo4yq+rWK6g3xiKgIMTnL+Esr/w2iJt7Sg8
+CGBRbQf5AC8htLwR1leNH7zqKk8AkFEm6bTPRgUWXvm3jFEvN2cJODjzyIkl/j/g
+xM7AQKlG6lOdIpSa8zO1WfkKVaKBfw3uJqEQ0pJ1bIf2W5/WvFm329V6Qc5tfZpz
+tb2eghDVCBXQsb1uKq5nyLt+Om3frLV3MHCae2yz9L5lZyuQ6RiI5s6O5/e5xSVD
+t5BcpGyXFqUyMFknO47J2pPh5MMVwGFmZs9LAAQywEASt+zbmAqrA8bzomwtl+GN
+KEcWLrfEdJ1GFzrShwb0eVIPZ4yKL/ZbFfGnCJu7Pa9H4oJx4BTl5kwzCIyuOvS8
+F39L3dVsurWcYzUtmThvnsnObChwslT+nRJfMWB+ov6Hj5+T5s0q9ttHKJlJ8uTV
+ZFHsA2KOtwPdG0OF97XXiP17D2Q14YEAt+vsChSoMullC0IWU+BKcTG4mDRfCI06
++sP+ZpNaDk+P50XnXG+/8MZWYi+a/wyZTFbj+85X6p9wryyJaYsuhNm9XOy1Ppr1
+Z5KtC0Kw/sit9KdyUlcGm5uhnmP7ZwIDAQABAoICAAa6TWDYNqopk22GJUJLaexS
+YtJn2VJ9ncB9ISUbV12jbVZuJoYTNy432aBIU+y7NoQd58mnirWMs2vqHMYPVTLW
+JA8fOWWFW5YK/imFgXpO0EAq8J6+Cyj3OeBAIIQB5QXXn4GiR96WCmoxx5i+2LSU
++fO54ykddcoFD2v7poiZKdr/XkAkwkOhIbWEnpvPzM2zA7+DdltDNCcHoMcHE7tY
+IxKJLpcAdV1gqUdZ1CksznJC1ZkrkVK7Do3JG6Gsjar7P65z99j+bol3j81Z5Fxa
+oAMj1cuBHh4InXmVQ0A1ac6QSAnvHHGa9JtuSS+1NnQ2NuDV0e086mKS1eL+Av60
+0upMs2wWHn20FclHfF3dPXGXW5U7D1DqY4Zy9p77Qoef8naeVWdruCoMGXEyv85w
+H2ZDfeQgjnRQTiWinEvkhJfi+qXQvC689rOtxhUX3il8FWLExV+Hix2K64c5Ne9P
+wCOflEQxwqBM3O259N9xaaKy5+fsMDoADzwQZtVEOZQbtPfIFIoZKhijSNpf72eb
+MUjroZVWl4ZSniskMf2ZzMtnVCho98VIpX9fh80yGSQ1QGpp/XnzfhpBFNvLH7O1
+NA3+s6XxXVBve/VU5JDW8FEf7UxmppON6q3+Vcaq3YVMP8jpedWY7yAQriGX0gAZ
+Bnqo18kH6U0RntFmOGrhAoIBAQD/MtH/tmA/1kuMnKD45Yu+XNtFdzljhelVxOx0
+EJX28ZE1X/pqOvbPUvTFa6EPULnfiajgzE622IQDCYgwkXM4yrn7R+zFA1WBq7O1
+dgbWZCdDA6RCpGK7/oSbOviPpovD3pfUkq9rhvh1VeGGgv0P3zZ5yZX6LUJtjyo6
+tk/1x4FHzPF4wcSgT2ZoQn+NpVBoSc6/8uUTmYSWjxq/wnhK7KZvbeqwhJ9ikFs/
+II5fmoqYwEyQbZQQNKgz1mpW+ZRurf2iEmyEjrY+E4mizqZhnXa8X6Nzn4B6wSeJ
+wlK283PTh3NoTwXklTCqLXv1MkIrtE97lXgCEKR7JKfeM8v3AoIBAQDPXGXBxZu2
+KReej0YMrEoVyKyjqdnYb+MyBPNRiL7czlfMeKscldG2e+Z8mkBpVWBIw6KgY8JK
+DPg7aziC9f0QUOgWKH33+9YN7xk4PSlkwQZhKyu9IQcYY2xo0LrptVtrzdn7/CyQ
+ApdEq2xe6dB+r2Cd3vM1+2dhkuATZK87Nlm3kXTiz7K5qA9nawnzL2+u/rF45izu
+kZezXMMpsKW6xxiePPyItVR+4Z4EZhg6kB/QNPkA8tjGptsgVBbiLJM6H6/xtfuf
+Mwjh0Yv0jzHiTly+AJL9rx3y5pPqg2rULOSbWRxRLkx75q8KLwfOOMTf+OwtKjyZ
+r8MchIkMbBARAoIBADlKtnx6/Ca4vGNH8peOKQ5GmG+C8Z5XPOglepQf+RrkZp4d
++wEIVcp7rDn6DMF8dQ4rQH+fPnisKQ7pf+qvbLeuQ4yXPB+KvRKMcp7lbWmKOIpB
+8gmIECZ2YFzdI1pUoIILofh2Ke2w8mydKDFjjN6YVQmIaSQuLwCbqHZf4Zmi/XIa
+H4flsHfw+2OisjIhj+ip0UGkjSsWRv7qB65PQWRItqDDg3G3hHTDRcjpTS1Ha6AZ
+Y9b00s3ElJJ2q471Hw6t/wf4rOYhh+ZtynODgzTc/gASVIaro1Nrs62os5shErrF
+aPJc80y69Z7u8So96z8WjtWG29dS1ypSM2GeLUcCggEBAIYhJlEpGYfDHNwboRwh
+deqRW9qhy6AM/9EjEqDy60K41mIUy9o5ruVzT6vZu0BnUVi/8zn8TXjI2ujUekF8
+DK25J+btWk5GQDfTKWUPau8ZTJ8d5bT44DYOWdmS6tSx0ujwxsgQXmLoyiBJIlhi
+tdK8bqqvxHJupHihIQBqaE7M4Uu0cv8jimA9LXmf61e6n2t6pCGoAfhvhMkof7U/
+5nPixTHWESP85yMLncMKpzF6eJmdKlRKwZ394FARFJxIaRN3279mD9TylhQ8D2Oq
+HIJeXe8pP+uIkr7EF3nid/+26kjyYza/1AlxNlhIA6yJXA/kXCD66SggYPzZXi0C
+2YECggEAAl9JXmVcoOEEXx7QoIRfFy8Tbz8dRpID7LInkuFWFc9vpLoCzAIajq78
+wxSQqnvcfUBGQn6ChmdIjSDgi5RJQ9ZxS0Pyze11a8RrXl295jyNJTPB/qMt4ECs
+ZwI7yGzk1veDEett6UCZMsFA3SUUTgR9epZSYtg+pGV7cluDGRwj8HcZxPOCgtBB
+UcZYd4PS/yOufVjujUVgdaYFLvk6y4WTZB2/HX99MzrHBFWP9NpJ7Ha5w55JcaS/
+XPKFeZLEPWizePuRvANCHoElEUm5DUlHcT4RwSHozGZwNhJhTOs/797p9FxXJKX7
+o2YTBJCBe5ojou7zXimYDgFm+By2AA==
+-----END PRIVATE KEY-----