Skip to content

Commit c0dd4c0

Browse files
github-roushangithub-roushan
authored
Merge branch 'janhq:dev' into dev
2 parents fdf976f + 76b1983 commit c0dd4c0

22 files changed

+616
-205
lines changed

‎BUILDING.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Build Cortex.cpp from source
2+
3+
Firstly, clone the Cortex.cpp repository [here](https://github.com/janhq/cortex.cpp) and initialize the submodules:
4+
5+
```bash
6+
git clone https://github.com/janhq/cortex.cpp
7+
cd cortex.cpp
8+
git submodule update --init --recursive
9+
```
10+
11+
You also need to install CMake. On Linux and MacOS, you can install CMake via your package manager
12+
13+
```bash
14+
sudo apt install cmake # Ubuntu
15+
brew install cmake # MacOS
16+
```
17+
18+
On Windows, you can download CMake from https://cmake.org/download/.
19+
20+
#### Windows
21+
22+
1. Navigate to the `engine` folder.
23+
2. Configure the vpkg:
24+
25+
```bash
26+
cd vcpkg
27+
./bootstrap-vcpkg.bat
28+
./vcpkg install
29+
```
30+
31+
3. Build the Cortex.cpp inside the `engine/build` folder (you can change `-DCMAKE_TOOLCHAIN_FILE` to use your own `vcpkg`):
32+
33+
```bash
34+
mkdir build
35+
cd build
36+
cmake .. -DBUILD_SHARED_LIBS=OFF -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows-static
37+
cmake --build . --config Release
38+
```
39+
40+
4. Verify that Cortex.cpp is installed correctly by getting help information.
41+
42+
```sh
43+
cortex -h
44+
```
45+
46+
#### Linux and MacOS
47+
48+
1. Navigate to the `engine` folder.
49+
2. Configure the vpkg:
50+
51+
```bash
52+
cd vcpkg
53+
./bootstrap-vcpkg.sh
54+
./vcpkg install
55+
```
56+
57+
3. Build the Cortex.cpp inside the `engine/build` folder (you can change `-DCMAKE_TOOLCHAIN_FILE` to use your own `vcpkg`):
58+
59+
```bash
60+
mkdir build
61+
cd build
62+
cmake .. -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake
63+
make -j4
64+
```
65+
66+
4. Verify that Cortex.cpp is installed correctly by getting help information.
67+
68+
```sh
69+
./cortex -h
70+
```
71+
72+
#### Devcontainer / Codespaces
73+
74+
1. Open Cortex.cpp repository in Codespaces or local devcontainer
75+
76+
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/janhq/cortex.cpp?quickstart=1)
77+
78+
```sh
79+
devcontainer up --workspace-folder .
80+
```
81+
82+
2. Configure vpkg in `engine/vcpkg`:
83+
84+
```bash {"tag": "devcontainer"}
85+
cd engine/vcpkg
86+
export VCPKG_FORCE_SYSTEM_BINARIES="$([[ $(uname -m) == 'arm64' ]] && echo '1' || echo '0')"
87+
./bootstrap-vcpkg.sh
88+
```
89+
90+
3. Build the Cortex.cpp inside the `engine/build` folder:
91+
92+
```bash {"tag": "devcontainer"}
93+
cd engine
94+
mkdir -p build
95+
cd build
96+
cmake .. -DCMAKE_TOOLCHAIN_FILE=$(realpath ..)/vcpkg/scripts/buildsystems/vcpkg.cmake
97+
make -j$(grep -c ^processor /proc/cpuinfo)
98+
```
99+
100+
4. Verify that Cortex.cpp is installed correctly by getting help information.
101+
102+
```sh {"tag": "devcontainer"}
103+
cd engine/build
104+
./cortex -h
105+
```
106+
107+
5. Everytime a rebuild is needed, just run the commands above using oneliner
108+
109+
```sh
110+
npx -y runme run --filename README.md -t devcontainer -y
111+
```

‎README.md‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -160,11 +160,7 @@ cortex-nightly hardware activate
160160

161161
### Build from Source
162162

163-
```bash
164-
git clone https://github.com/janhq/cortex.cpp
165-
cd engine/vcpkg && ./bootstrap-vcpkg.sh
166-
cd ../build && cmake .. && make -j4
167-
```
163+
See [BUILDING.md](BUILDING.md)
168164

169165
## Uninstall Cortex
170166

‎engine/cli/commands/server_start_cmd.cc‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -80,17 +80,17 @@ bool ServerStartCmd::Exec(const std::string& host, int port,
8080
mutable_cmds.push_back(L'\0');
8181
// Create child process
8282
if (!CreateProcess(
83-
NULL, // No module name (use command line)
83+
NULL, // No module name (use command line)
8484
mutable_cmds
85-
.data(), // Command line (replace with your actual executable)
86-
NULL, // Process handle not inheritable
87-
NULL, // Thread handle not inheritable
88-
FALSE, // Set handle inheritance
89-
0, // No creation flags
90-
NULL, // Use parent's environment block
91-
NULL, // Use parent's starting directory
92-
&si, // Pointer to STARTUPINFO structure
93-
&pi)) // Pointer to PROCESS_INFORMATION structure
85+
.data(), // Command line (replace with your actual executable)
86+
NULL, // Process handle not inheritable
87+
NULL, // Thread handle not inheritable
88+
FALSE, // Set handle inheritance
89+
CREATE_NO_WINDOW, // No new console
90+
NULL, // Use parent's environment block
91+
NULL, // Use parent's starting directory
92+
&si, // Pointer to STARTUPINFO structure
93+
&pi)) // Pointer to PROCESS_INFORMATION structure
9494
{
9595
std::cout << "Could not start server: " << GetLastError() << std::endl;
9696
return false;

‎engine/controllers/messages.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ class Messages : public drogon::HttpController<Messages, false> {
2020

2121
ADD_METHOD_TO(Messages::RetrieveMessage, "/v1/threads/{1}/messages/{2}", Get);
2222
ADD_METHOD_TO(Messages::ModifyMessage, "/v1/threads/{1}/messages/{2}",
23-
Options, Post);
23+
Options, Patch);
2424
ADD_METHOD_TO(Messages::DeleteMessage, "/v1/threads/{1}/messages/{2}",
2525
Options, Delete);
2626
METHOD_LIST_END

‎engine/controllers/models.cc‎

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,20 @@ void Models::PullModel(const HttpRequestPtr& req,
5858
model_handle, desired_model_id, desired_model_name);
5959
} else if (model_handle.find(":") != std::string::npos) {
6060
auto model_and_branch = string_utils::SplitBy(model_handle, ":");
61+
if (model_and_branch.size() == 3) {
62+
auto mh = url_parser::Url{
63+
.protocol = "https",
64+
.host = kHuggingFaceHost,
65+
.pathParams = {
66+
model_and_branch[0],
67+
model_and_branch[1],
68+
"resolve",
69+
"main",
70+
model_and_branch[2],
71+
}}.ToFullPath();
72+
return model_service_->HandleDownloadUrlAsync(mh, desired_model_id,
73+
desired_model_name);
74+
}
6175
return model_service_->DownloadModelFromCortexsoAsync(
6276
model_and_branch[0], model_and_branch[1], desired_model_id);
6377
}
@@ -813,15 +827,34 @@ void Models::GetModelSources(
813827
resp->setStatusCode(k400BadRequest);
814828
callback(resp);
815829
} else {
816-
auto const& info = res.value();
830+
auto& info = res.value();
817831
Json::Value ret;
818832
Json::Value data(Json::arrayValue);
819-
for (auto const& i : info) {
820-
data.append(i);
833+
for (auto& i : info) {
834+
data.append(i.second.ToJson());
821835
}
822836
ret["data"] = data;
823837
auto resp = cortex_utils::CreateCortexHttpJsonResponse(ret);
824838
resp->setStatusCode(k200OK);
825839
callback(resp);
826840
}
841+
}
842+
843+
void Models::GetModelSource(
844+
const HttpRequestPtr& req,
845+
std::function<void(const HttpResponsePtr&)>&& callback,
846+
const std::string& src) {
847+
auto res = model_src_svc_->GetModelSource(src);
848+
if (res.has_error()) {
849+
Json::Value ret;
850+
ret["message"] = res.error();
851+
auto resp = cortex_utils::CreateCortexHttpJsonResponse(ret);
852+
resp->setStatusCode(k400BadRequest);
853+
callback(resp);
854+
} else {
855+
auto& info = res.value();
856+
auto resp = cortex_utils::CreateCortexHttpJsonResponse(info.ToJson());
857+
resp->setStatusCode(k200OK);
858+
callback(resp);
859+
}
827860
}

‎engine/controllers/models.h‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ class Models : public drogon::HttpController<Models, false> {
4343
ADD_METHOD_TO(Models::AddModelSource, "/v1/models/sources", Post);
4444
ADD_METHOD_TO(Models::DeleteModelSource, "/v1/models/sources", Delete);
4545
ADD_METHOD_TO(Models::GetModelSources, "/v1/models/sources", Get);
46+
ADD_METHOD_TO(Models::GetModelSource, "/v1/models/sources/{src}", Get);
4647
METHOD_LIST_END
4748

4849
explicit Models(std::shared_ptr<DatabaseService> db_service,
@@ -106,6 +107,10 @@ class Models : public drogon::HttpController<Models, false> {
106107
void GetModelSources(const HttpRequestPtr& req,
107108
std::function<void(const HttpResponsePtr&)>&& callback);
108109

110+
void GetModelSource(const HttpRequestPtr& req,
111+
std::function<void(const HttpResponsePtr&)>&& callback,
112+
const std::string& src);
113+
109114
private:
110115
std::shared_ptr<DatabaseService> db_service_;
111116
std::shared_ptr<ModelService> model_service_;

‎engine/controllers/threads.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ class Threads : public drogon::HttpController<Threads, false> {
2020

2121
ADD_METHOD_TO(Threads::RetrieveThread, "/v1/threads/{thread_id}", Get);
2222
ADD_METHOD_TO(Threads::ModifyThread, "/v1/threads/{thread_id}", Options,
23-
Post);
23+
Patch);
2424
ADD_METHOD_TO(Threads::DeleteThread, "/v1/threads/{thread_id}", Options,
2525
Delete);
2626
METHOD_LIST_END

‎engine/database/models.cc‎

Lines changed: 45 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -270,35 +270,63 @@ bool Models::HasModel(const std::string& identifier) const {
270270
}
271271
}
272272

273-
cpp::result<std::vector<std::string>, std::string> Models::GetModelSources()
274-
const {
273+
cpp::result<std::vector<ModelEntry>, std::string> Models::GetModels(
274+
const std::string& model_src) const {
275275
try {
276-
std::vector<std::string> sources;
276+
std::vector<ModelEntry> res;
277277
SQLite::Statement query(db_,
278-
"SELECT DISTINCT model_source FROM models WHERE "
279-
"status = \"downloadable\"");
280-
278+
"SELECT model_id, author_repo_id, branch_name, "
279+
"path_to_model_yaml, model_alias, model_format, "
280+
"model_source, status, engine, metadata FROM "
281+
"models WHERE model_source = "
282+
"? AND status = \"downloadable\"");
283+
query.bind(1, model_src);
281284
while (query.executeStep()) {
282-
sources.push_back(query.getColumn(0).getString());
285+
ModelEntry entry;
286+
entry.model = query.getColumn(0).getString();
287+
entry.author_repo_id = query.getColumn(1).getString();
288+
entry.branch_name = query.getColumn(2).getString();
289+
entry.path_to_model_yaml = query.getColumn(3).getString();
290+
entry.model_alias = query.getColumn(4).getString();
291+
entry.model_format = query.getColumn(5).getString();
292+
entry.model_source = query.getColumn(6).getString();
293+
entry.status = StringToStatus(query.getColumn(7).getString());
294+
entry.engine = query.getColumn(8).getString();
295+
entry.metadata = query.getColumn(9).getString();
296+
res.push_back(entry);
283297
}
284-
return sources;
298+
return res;
285299
} catch (const std::exception& e) {
286300
return cpp::fail(e.what());
287301
}
288302
}
289303

290-
cpp::result<std::vector<std::string>, std::string> Models::GetModels(
291-
const std::string& model_src) const {
304+
cpp::result<std::vector<ModelEntry>, std::string> Models::GetModelSources()
305+
const {
292306
try {
293-
std::vector<std::string> ids;
294-
SQLite::Statement query(db_,
295-
"SELECT model_id FROM models WHERE model_source = "
296-
"? AND status = \"downloadable\"");
297-
query.bind(1, model_src);
307+
std::vector<ModelEntry> res;
308+
SQLite::Statement query(
309+
db_,
310+
"SELECT model_id, author_repo_id, branch_name, "
311+
"path_to_model_yaml, model_alias, model_format, "
312+
"model_source, status, engine, metadata FROM models "
313+
"WHERE model_source != \"\" AND (status = \"downloaded\" OR status = "
314+
"\"downloadable\")");
298315
while (query.executeStep()) {
299-
ids.push_back(query.getColumn(0).getString());
316+
ModelEntry entry;
317+
entry.model = query.getColumn(0).getString();
318+
entry.author_repo_id = query.getColumn(1).getString();
319+
entry.branch_name = query.getColumn(2).getString();
320+
entry.path_to_model_yaml = query.getColumn(3).getString();
321+
entry.model_alias = query.getColumn(4).getString();
322+
entry.model_format = query.getColumn(5).getString();
323+
entry.model_source = query.getColumn(6).getString();
324+
entry.status = StringToStatus(query.getColumn(7).getString());
325+
entry.engine = query.getColumn(8).getString();
326+
entry.metadata = query.getColumn(9).getString();
327+
res.push_back(entry);
300328
}
301-
return ids;
329+
return res;
302330
} catch (const std::exception& e) {
303331
return cpp::fail(e.what());
304332
}

‎engine/database/models.h‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ namespace cortex::db {
1010

1111
enum class ModelStatus { Remote, Downloaded, Downloadable };
1212

13-
1413
struct ModelEntry {
1514
std::string model;
1615
std::string author_repo_id;
@@ -57,9 +56,9 @@ class Models {
5756
cpp::result<std::vector<std::string>, std::string> FindRelatedModel(
5857
const std::string& identifier) const;
5958
bool HasModel(const std::string& identifier) const;
60-
cpp::result<std::vector<std::string>, std::string> GetModelSources() const;
61-
cpp::result<std::vector<std::string>, std::string> GetModels(
59+
cpp::result<std::vector<ModelEntry>, std::string> GetModels(
6260
const std::string& model_src) const;
61+
cpp::result<std::vector<ModelEntry>, std::string> GetModelSources() const;
6362
};
6463

6564
} // namespace cortex::db

0 commit comments

Comments
 (0)