Skip to content

10. Saving Requests to Disk

In this lesson, we will modify our code to save all requests and their corresponding responses to a specified directory instead of only overriding the requests sent to OpenAI.

Refactoring chatgpt-send with chatgpt-request

To improve clarity and organization, we refactor our code to separate the request generation process into a dedicated function, chatgpt-request.

(defun chatgpt-request (prompt)
  "Return an OpenAI request with PROMPT."
  `(:model "gpt-4o"
    :messages ,(vector `(:role "user" :content ,prompt))))

(defun chatgpt-send ()
  "Send a request to OpenAI."
  (interactive)
  (let* ((json-encoding-pretty-print t)
         (prompt (buffer-string))
         (req (chatgpt-request prompt))
         (req-path "/home/tony/chatgpt-emacs/request.json")
         (command (chatgpt-command req-path)))
    (write-region (json-encode req) nil req-path)
    (make-process ...)))

Refactoring chatgpt-send with chatgpt-callback

Next, we enhance our chatgpt-send function by creating another function, chatgpt-callback, to manage appending prompts and responses to the *chatgpt[requests]* buffer.

(defun chatgpt-callback (prompt response)
  "Append PROMPT and RESPONSE to the prompt buffer."
  (with-current-buffer (get-buffer-create "*chatgpt[requests]*")
    (markdown-mode)
    (goto-char (point-max))
    (insert "# Request\n\n"
            "## Prompt\n\n" prompt "\n\n"
            "## Response\n\n" response "\n\n")))

(defun chatgpt-send ()
  "Send a request to OpenAI."
  (interactive)
  (let* (...)
    (write-region (json-encode req) nil req-path)
    (make-process
     ...
     :sentinel
     (lambda (process event)
       (if (not (string= event "finished\n"))
           (message "Error")
         (let* (...)
           (chatgpt-callback prompt response))
         (kill-buffer (process-buffer process)))))))

Saving Requests

In this section, we enhance the chatgpt-send command to save each request sent to OpenAI in a unique subdirectory along with its corresponding JSON response.

Before we modify the chatgpt-send function, let's review a couple of useful Emacs Lisp functions we'll employ later.

The make-temp-file function allows us to create unique subdirectories under the temporary-file-directory. For example:

(make-temp-file nil t) ;; "/tmp/5ukiOy"

This expression generates a subdirectory named 5ukiOy within /tmp/, the current temporary-file-directory.

To ensure the path returned by make-temp-file ends with a forward slash, we can use the file-name-as-directory function:

(file-name-as-directory (make-temp-file nil t)) ;; "/tmp/1I39B7/"

Additionally, we can temporarily set the temporary-file-directory to a specified existing directory, allowing subdirectories to be relative to that path:

(make-directory "/tmp/foo/bar/" t)
(let ((temporary-file-directory "/tmp/foo/bar/"))
  (file-name-as-directory (make-temp-file nil t)))
;; "/tmp/foo/bar/LcMrzD/"

In this snippet, we used make-directory to create /tmp/foo/bar/, with the t parameter enabling the creation of any missing parent directories.

Now, let's integrate this into our package. We introduce the chatgpt-dir variable to specify the requests directory. Within the chatgpt-send function, we bind temporary-file-directory to chatgpt-dir. We then create a new subdirectory, assigning its path to the req-dir variable and defining req-path as the request.json file within req-dir. We also ensure the existence of the chatgpt-dir. Below is the updated chatgpt-send function:

(defvar chatgpt-dir "/home/tony/chatgpt-emacs/requests/"
  "Request directory.")

(defun chatgpt-send ()
  "Send a request to OpenAI."
  (interactive)
  (make-directory chatgpt-dir t)
  (let* (...
         (req (chatgpt-request prompt))
         (temporary-file-directory chatgpt-dir)
         (req-dir (file-name-as-directory (make-temp-file nil t)))
         (req-path (concat req-dir "request.json"))
         ...)
    (message "chatgpt: %s" req-dir)
    (write-region (json-encode req) nil req-path)
    (make-process ...)))

When invoking chatgpt-send with the prompt "Hello!", the request is saved in the file /home/tony/chatgpt-emacs/requests/U4v02L/request.json as follows:

{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}

Additionally, the following prompt and response are appended to the *chatgpt[requests]* buffer:

# Request

## Prompt

Hello!

## Response

Hello! How can I assist you today?

Saving Responses

Next, we implement functionality to write the JSON responses to response.json files within the same directories as their corresponding requests.

To accomplish this, we adjust the sentinel function binding resp-path to point to the response.json file located in req-dir. The resp response is encoded and written similarly to the requests, using json-encoding-pretty-print set to t for improved readability.

(lambda (process event)
  (if (not (string= event "finished\n"))
      (message "Error")
    (let* ((resp (with-current-buffer (process-buffer process)
                   (goto-char (point-min))
                   (chatgpt-json-read)))
           (response (map-nested-elt resp [:choices 0 :message :content]))
           (resp-path (concat req-dir "response.json"))
           (json-encoding-pretty-print t))
      (write-region (json-encode resp) nil resp-path)
      (chatgpt-callback prompt response))
    (kill-buffer (process-buffer process))))

When executing chatgpt-send with the "Hello!" prompt, it stores the corresponding response in /home/tony/chatgpt-emacs/requests/0xI4Yw/response.json.

Refactoring chatgpt-send with chatgpt-json-encode

To streamline our encoding process, we define a new function, chatgpt-json-encode, to handle JSON encoding with pretty printing.

(defun chatgpt-json-encode (object)
  (let ((json-encoding-pretty-print t))
    (json-encode object)))

(defun chatgpt-send ()
  "Send a request to OpenAI."
  (interactive)
  (make-directory chatgpt-dir t)
  (let* (...)
    (message "chatgpt: %s" req-dir)
    (write-region (chatgpt-json-encode req) nil req-path)
    (make-process
     ...
     :sentinel
     (lambda (process event)
       (if (not (string= event "finished\n"))
           (message "Error")
         (let* (...)
           (write-region (chatgpt-json-encode resp) nil resp-path)
           (chatgpt-callback prompt response))
         (kill-buffer (process-buffer process)))))))

Finally, we update the chatgpt-callback function to include links to the request directories when appending prompts and responses to the buffer.

(defun chatgpt-callback (prompt response req-dir)
  "Append PROMPT and RESPONSE to the prompt buffer with a link to REQ-DIR."
  (with-current-buffer (get-buffer-create "*chatgpt[requests]*")
    (markdown-mode)
    (goto-char (point-max))
    (insert "# Request\n\n"
            "<!-- [](" req-dir ") -->\n\n"
            "## Prompt\n\n" prompt "\n\n"
            "## Response\n\n" response "\n\n")))

We then pass the req-dir argument correctly to chatgpt-callback function in the sentinel function:

(defun chatgpt-send ()
  ...
  (let* (...
         (temporary-file-directory chatgpt-dir)
         (req-dir (file-name-as-directory (make-temp-file nil t)))
         ...)
    ...
    (make-process
     ...
     :sentinel
     (lambda (process event)
       (if (not (string= event "finished\n"))
           (message "Error")
         (let* (...)
           (write-region (chatgpt-json-encode resp) nil resp-path)
           (chatgpt-callback prompt response req-dir))
         (kill-buffer (process-buffer process)))))))

When executing chatgpt-send with the prompt "Hello!", the buffer *chatgpt[requests]* includes a link to the request directory:

# Request

<!-- [](/home/tony/chatgpt-emacs/requests/Xb7cAy/) -->

## Prompt

Hello!

## Response

Hi there! How can I assist you today?

In this lesson, we have learned to save all requests and their corresponding responses to disk. In the next lesson, we will implement a command to display the prompt buffer at the bottom of the frame.