Skip to content

Commit

Permalink
rpc : get available mem for the CPU backend
Browse files Browse the repository at this point in the history 
This can be overridden with the -m command line option

ref: #7293
  • Loading branch information
@rgerganov
rgerganov committed May 16, 2024
1 parent 3b3963c commit 9afdffe
Showing 1 changed file with 17 additions and 3 deletions.
20 changes: 17 additions & 3 deletions examples/rpc/rpc-server.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
#endif

#include "ggml-rpc.h"
#ifdef _WIN32
# include <windows.h>
#else
# include <unistd.h>
#endif
#include <string>
#include <stdio.h>

Expand Down Expand Up @@ -84,9 +89,18 @@ static void get_backend_memory(size_t * free_mem, size_t * total_mem) {
#ifdef GGML_USE_CUDA
ggml_backend_cuda_get_device_memory(0, free_mem, total_mem);
#else
// TODO: implement for other backends
*free_mem = 1;
*total_mem = 1;
#ifdef _WIN32
MEMORYSTATUSEX status;
status.dwLength = sizeof(status);
GlobalMemoryStatusEx(&status);
*total_mem = status.ullTotalPhys;
*free_mem = status.ullAvailPhys;
#else
long pages = sysconf(_SC_PHYS_PAGES);
long page_size = sysconf(_SC_PAGE_SIZE);
*total_mem = pages * page_size;
*free_mem = *total_mem;
#endif
#endif
}

Expand Down

2 comments on commit 9afdffe

@ggerganov
Copy link
Owner

@ggerganov ggerganov commented on 9afdffe May 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The (#PR) at the end of the commit message subject are useful for the sync scripts to keep track in which repo the commit originated from:

llama.cpp/scripts/sync-ggml-am.sh

Lines 84 to 92 in 3b3963c

# replace PR numbers
#
# Subject: some text (#1234)
# Subject: some text (ggml/1234)
cat ggml-src.patch | sed -e 's/^Subject: \(.*\) (#\([0-9]*\))/Subject: \1 (ggml\/\2)/' > ggml-src.patch.tmp
mv ggml-src.patch.tmp ggml-src.patch
cat ggml-src.patch | sed -e 's/^\(.*\) (#\([0-9]*\))$/\1 (ggml\/\2)/' > ggml-src.patch.tmp
mv ggml-src.patch.tmp ggml-src.patch

These are automatically added by the Github UI when squash merging, even for single commits

@rgerganov
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I wondered what is the difference between "squash and merge" and "rebase and merge" for single commits. From now on I will use only "squash and merge"

Please sign in to comment.