Skip to content

gptel-gh: Change how tokens are managed#1058

Open
marcus-lundgren wants to merge 16 commits intokarthink:masterfrom
marcus-lundgren:gh-change-token-management
Open

gptel-gh: Change how tokens are managed#1058
marcus-lundgren wants to merge 16 commits intokarthink:masterfrom
marcus-lundgren:gh-change-token-management

Conversation

@marcus-lundgren
Copy link
Copy Markdown

@marcus-lundgren marcus-lundgren commented Sep 3, 2025

Removed the file based persistent storage of the tokens, as they were stored in clear text. They will only be stored in variables.

The initial value of the github-token can now be set by providing a function that provides it. This enables this token to be securely stored outside of Emacs.

@marcus-lundgren marcus-lundgren marked this pull request as draft September 3, 2025 20:56
@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 68b1fc9 to 76a1010 Compare September 9, 2025 21:16
@marcus-lundgren
Copy link
Copy Markdown
Author

@karthink - Do you have any input in how you imagine the workflow to be for this API regarding the GitHub token?

With the changes made just recently to present the login function, is it reasonable to have a workflow to perform that in order to get the token that can then be stored elsewhere?

@karthink
Copy link
Copy Markdown
Owner

karthink commented Sep 9, 2025 via email

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from b31c107 to 7fa9017 Compare September 15, 2025 11:51
@marcus-lundgren
Copy link
Copy Markdown
Author

@kiennq - A friendly reminder about this PR :)

@kiennq
Copy link
Copy Markdown
Contributor

kiennq commented Sep 23, 2025

Sorry for the late reply. Can you add a default/example for github-token-init? At least it should be defaulted to the current workflow where the token is saved/restored from a file, else we're looking at reauth everytime gptel is reloaded

@marcus-lundgren
Copy link
Copy Markdown
Author

Thanks for your reply, @kiennq!

This is what I am currently using:

(use-package gptel
  :defer t
  :config (setq
           gptel-backend (gptel-make-gh-copilot "Copilot"
                           :github-token-init (lambda () (read-passwd "Copilot token: "))))
  )

I store my token in a password manager, so I use read-passwd in order to enter it when calling gptel. I will only be prompted once per Emacs session for this, which is a very reasonable workflow for me as I rarely restart Emacs. As it takes just a function, it is trivial to make it read from a file instead.

With the current implementation in this PR, the login function is not able to persistently store the newly acquired token. The new token is only stored in a variable and I take the it from the message log, which is not an ideal scenario and hence my request for feedback earlier. Another approach could be to introduce load and save functions that can be given as parameters to gptel-make-gh-copilot. It would then allow the user to choose how it should be stored and saved.

Before making the changes in gptel itself, I used the code below as a patch to control how the token was stored. This also has the behavior of only requesting the token once. As you may notice, the logic for the chat message token was kept, but this was before me realising that that is also a thing to keep secret.

(use-package gptel
  :config
  (progn
    (defvar my/gh-token nil "Store the Copilot token for GPTel commands.")
    (defalias 'original-gptel--gh-save (symbol-function 'gptel--gh-save))
    (defalias 'original-gptel--gh-restore (symbol-function 'gptel--gh-restore))
    (defun gptel--gh-restore (file)
      (if (equal file gptel-gh-github-token-file)
          (if my/gh-token
              my/gh-token
            (progn 
              (setq my/gh-token
                    (let ((token (read-passwd "Copilot token: ")))
                      (if (string= "" token) nil token)))
              my/gh-token))
        (original-gptel--gh-restore file)))
    (defun gptel--gh-save (file obj)
      (message "New GH token: %s" obj)
      (if (equal file gptel-gh-github-token-file)
          (setq my/gh-token obj)
        (original-gptel--gh-save file obj)))))

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 7fa9017 to c293cbf Compare October 4, 2025 19:40
@marcus-lundgren
Copy link
Copy Markdown
Author

@kiennq - Any thoughts? :)

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from c293cbf to 63c1bec Compare October 11, 2025 20:11
@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 63c1bec to 83b7a84 Compare October 22, 2025 18:33
@marcus-lundgren
Copy link
Copy Markdown
Author

marcus-lundgren commented Oct 22, 2025

@kiennq - I've now changed the implementation so that it defaults to use the file as before for the github-token.

What has been additionally added is the ability to create a custom save function for the github-token. This solves the question that I raised before, as it is now up to the one redefining the save function to decide how to handle an update.

A sample use-package snippet to override it:

(use-package gptel
  :defer t
  :config
  (defvar my/gh-token nil "Store the Copilot token for GPTel commands.")
  (setq
   gptel-gh-github-token-load (lambda ()
                                (if my/gh-token
                                    my/gh-token
                                  (progn 
                                    (setq my/gh-token
                                          (let ((token (read-passwd "Copilot token: ")))
                                            (if (string= "" token) nil token)))
                                    my/gh-token)))
   gptel-gh-github-token-save (lambda (token)
                                (setq my/gh-token token))
   gptel-backend (gptel-make-gh-copilot "Copilot"))
  )

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from c0727e8 to 0bc293c Compare October 27, 2025 21:59
@marcus-lundgren
Copy link
Copy Markdown
Author

@kiennq - a friendly reminder concerning this. I think the current solution in this MR strikes a good balance between ease of use and choice of security. If you agree, then I can start working on changes to the README.

@karthink - do you have any thoughts regarding the latest proposal?

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch 4 times, most recently from 0ed7be3 to 51ba56e Compare November 5, 2025 21:01
@marcus-lundgren
Copy link
Copy Markdown
Author

@kiennq, @karthink - another reminder.

@karthink
Copy link
Copy Markdown
Owner

@marcus-lundgren just dropping a note to say this PR remains on my radar -- I just need a longer chunk of time to review it than other PRs since I'm not the author of the token management code. I'll do a general code review but I trust @kiennq's judgment on the design.

@marcus-lundgren
Copy link
Copy Markdown
Author

@karthink - thank you for the status update! As you may have noticed, I am rebasing the branch continuously to fix any conflicts (as happened a few days ago).

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 51ba56e to c67ae91 Compare November 16, 2025 21:26
@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from c67ae91 to fdfee2a Compare November 18, 2025 19:51
@marcus-lundgren
Copy link
Copy Markdown
Author

Updated config example with the builtin cache functionality:

(use-package gptel
  :defer t
  :config
  (setq
   gptel-gh-github-token-load-function (lambda () (read-passwd "Copilot token: "))
   gptel-gh-github-token-save-function (lambda (token) (message "Token saved: %s" token))
   gptel-backend (gptel-make-gh-copilot "Copilot"))
  )

@kiennq
Copy link
Copy Markdown
Contributor

kiennq commented Nov 20, 2025

I left one comment, else it LGTM

@marcus-lundgren
Copy link
Copy Markdown
Author

@karthink - Are you looking at the latest version? I have reverted from the global gptel--gh-token so that it is placed in each backend as before.

@karthink
Copy link
Copy Markdown
Owner

karthink commented Feb 23, 2026 via email

@karthink karthink force-pushed the gh-change-token-management branch from 74e66fd to 10088c0 Compare March 8, 2026 17:56
@karthink
Copy link
Copy Markdown
Owner

karthink commented Mar 8, 2026

I tried to test it as follows:

(gptel-make-gh-copilot "Copilot")
  • Run M-x gptel-gh-login
  • Authorize in the browser
  • After authorizing and pressing RET in Emacs, I got an error with this backtrace:
Debugger entered--Lisp error: (wrong-type-argument gptel--gh #s(gptel-gemini :name "Gemini" :host "generativelanguage.googleapis.com" :header nil :protocol "https" :stream t :endpoint "/v1beta/models" :key gptel-api-key-from-auth-source :models (gemini-pro-latest gemini-flash-latest gemini-flash-lite-latest) :url #f(lambda () [(endpoint "/v1beta/models") (protocol "https") (host "generativelanguage.googleapis.com") (stream t)] (let ((method (if (and stream gptel-use-curl gptel-stream) "streamGenerateContent" "generateContent"))) (format "%s://%s%s/%s:%s?key=%s" protocol host endpoint gptel-model method (gptel--get-api-key)))) :request-params nil :curl-args nil :coding-system nil))
  (signal wrong-type-argument (gptel--gh #s(gptel-gemini :name "Gemini" :host "generativelanguage.googleapis.com" :header nil :protocol "https" :stream t :endpoint "/v1beta/models" :key gptel-api-key-from-auth-source :models (gemini-pro-latest gemini-flash-latest gemini-flash-lite-latest) :url #f(lambda () [(endpoint "/v1beta/models") (protocol "https") (host "generativelanguage.googleapis.com") (stream t)] (let ((method (if ... "streamGenerateContent" "generateContent"))) (format "%s://%s%s/%s:%s?key=%s" protocol host endpoint gptel-model method (gptel--get-api-key)))) :request-params nil :curl-args nil :coding-system nil)))
  (or (let* ((cl-x cl-x)) (progn (and (memq (type-of cl-x) cl-struct-gptel--gh-tags) t))) (signal 'wrong-type-argument (list 'gptel--gh cl-x)))
  (let* ((cl-x gptel-backend)) (or (let* ((cl-x cl-x)) (progn (and (memq (type-of cl-x) cl-struct-gptel--gh-tags) t))) (signal 'wrong-type-argument (list 'gptel--gh cl-x))) (let* ((v cl-x)) (aset v 14 token)))
  (funcall gptel-gh-github-token-save-function github-username (let* ((cl-x gptel-backend)) (or (let* ((cl-x cl-x)) (progn (and (memq (type-of cl-x) cl-struct-gptel--gh-tags) t))) (signal 'wrong-type-argument (list 'gptel--gh cl-x))) (let* ((v cl-x)) (aset v 14 token))))
  (gptel--gh-save-github-token "" "ghu_XXXXXXXXXXXXXXXXXXXXXXXXXX")
  (let ((github-token (plist-get (gptel--url-retrieve "https://github.com/login/oauth/access_token" :method 'post :headers gptel--gh-auth-common-headers :data (cons ':client_id (cons gptel--gh-client-id (cons ... ...)))) :access_token))) (if (or (null github-token) (string-empty-p github-token)) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token))
  (progn (gui-set-selection 'CLIPBOARD user_code) (let ((username-text (if (string= github-username "") "Login for the default account." (format "Login for '%s'." github-username)))) (if in-ssh-session (progn (message "GitHub Device Code: %s (copied to clipboard)" user_code) (read-from-minibuffer (format "%s Code %s is copied. Visit %s in your local browser, enter the code, and authorize.  Press ENTER after authorizing. " username-text user_code verification_uri))) (read-from-minibuffer (format "%s Your one-time code %s is copied. Press ENTER to open GitHub in your browser. If your browser does not open automatically, browse to %s." username-text user_code verification_uri)) (browse-url verification_uri) (read-from-minibuffer "Press ENTER after authorizing."))) (let ((github-token (plist-get (gptel--url-retrieve "https://github.com/login/oauth/access_token" :method 'post :headers gptel--gh-auth-common-headers :data (cons ':client_id (cons gptel--gh-client-id ...))) :access_token))) (if (or (null github-token) (string-empty-p github-token)) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token)))
  (let ((device_code x1894) (user_code x1895) (verification_uri x1896)) (progn (gui-set-selection 'CLIPBOARD user_code) (let ((username-text (if (string= github-username "") "Login for the default account." (format "Login for '%s'." github-username)))) (if in-ssh-session (progn (message "GitHub Device Code: %s (copied to clipboard)" user_code) (read-from-minibuffer (format "%s Code %s is copied. Visit %s in your local browser, enter the code, and authorize.  Press ENTER after authorizing. " username-text user_code verification_uri))) (read-from-minibuffer (format "%s Your one-time code %s is copied. Press ENTER to open GitHub in your browser. If your browser does not open automatically, browse to %s." username-text user_code verification_uri)) (browse-url verification_uri) (read-from-minibuffer "Press ENTER after authorizing."))) (let ((github-token (plist-get (gptel--url-retrieve "https://github.com/login/oauth/access_token" :method 'post :headers gptel--gh-auth-common-headers :data (cons ... ...)) :access_token))) (if (or (null github-token) (string-empty-p github-token)) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token))))
  (let* ((x1894 (map-elt val :device_code)) (x1895 (map-elt val :user_code)) (x1896 (map-elt val :verification_uri))) (let ((device_code x1894) (user_code x1895) (verification_uri x1896)) (progn (gui-set-selection 'CLIPBOARD user_code) (let ((username-text (if (string= github-username "") "Login for the default account." (format "Login for '%s'." github-username)))) (if in-ssh-session (progn (message "GitHub Device Code: %s (copied to clipboard)" user_code) (read-from-minibuffer (format "%s Code %s is copied. Visit %s in your local browser, enter the code, and authorize.  Press ENTER after authorizing. " username-text user_code verification_uri))) (read-from-minibuffer (format "%s Your one-time code %s is copied. Press ENTER to open GitHub in your browser. If your browser does not open automatically, browse to %s." username-text user_code verification_uri)) (browse-url verification_uri) (read-from-minibuffer "Press ENTER after authorizing."))) (let ((github-token (plist-get (gptel--url-retrieve "https://github.com/login/oauth/access_token" :method ... :headers gptel--gh-auth-common-headers :data ...) :access_token))) (if (or (null github-token) (string-empty-p github-token)) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token)))))
  (progn (ignore (mapp val)) (let* ((x1894 (map-elt val :device_code)) (x1895 (map-elt val :user_code)) (x1896 (map-elt val :verification_uri))) (let ((device_code x1894) (user_code x1895) (verification_uri x1896)) (progn (gui-set-selection 'CLIPBOARD user_code) (let ((username-text (if ... "Login for the default account." ...))) (if in-ssh-session (progn (message "GitHub Device Code: %s (copied to clipboard)" user_code) (read-from-minibuffer ...)) (read-from-minibuffer (format "%s Your one-time code %s is copied. Press ENTER to open GitHub in your browser. If your browser does not open automatically, browse to %s." username-text user_code verification_uri)) (browse-url verification_uri) (read-from-minibuffer "Press ENTER after authorizing."))) (let ((github-token (plist-get ... :access_token))) (if (or (null github-token) (string-empty-p github-token)) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token))))))
  (let* ((val (gptel--url-retrieve "https://github.com/login/device/code" :method 'post :headers gptel--gh-auth-common-headers :data (cons ':client_id (cons gptel--gh-client-id '(:scope "read:user")))))) (progn (ignore (mapp val)) (let* ((x1894 (map-elt val :device_code)) (x1895 (map-elt val :user_code)) (x1896 (map-elt val :verification_uri))) (let ((device_code x1894) (user_code x1895) (verification_uri x1896)) (progn (gui-set-selection 'CLIPBOARD user_code) (let ((username-text ...)) (if in-ssh-session (progn ... ...) (read-from-minibuffer ...) (browse-url verification_uri) (read-from-minibuffer "Press ENTER after authorizing."))) (let ((github-token ...)) (if (or ... ...) (user-error "Error: You might not have access to GitHub Copilot Chat!")) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token)))))))
  (let ((gh-backends (gptel--gh-get-backends-by-username github-username)) (in-ssh-session (or (getenv "SSH_CLIENT") (getenv "SSH_CONNECTION") (getenv "SSH_TTY")))) (if (= (length gh-backends) 0) (user-error "No GitHub CoPilot backend found for username '%s'" github-username)) (let* ((val (gptel--url-retrieve "https://github.com/login/device/code" :method 'post :headers gptel--gh-auth-common-headers :data (cons ':client_id (cons gptel--gh-client-id '...))))) (progn (ignore (mapp val)) (let* ((x1894 (map-elt val :device_code)) (x1895 (map-elt val :user_code)) (x1896 (map-elt val :verification_uri))) (let ((device_code x1894) (user_code x1895) (verification_uri x1896)) (progn (gui-set-selection 'CLIPBOARD user_code) (let (...) (if in-ssh-session ... ... ... ...)) (let (...) (if ... ...) (message "Successfully logged in to GitHub Copilot") (gptel--gh-save-github-token github-username github-token))))))))
  (gptel-gh-login "")
  (funcall-interactively gptel-gh-login "")
  (command-execute gptel-gh-login record)
  (execute-extended-command nil "gptel-gh-login" "gptel-gh-login")
  (funcall-interactively execute-extended-command nil "gptel-gh-login" "gptel-gh-login")
  (command-execute execute-extended-command)

gptel--gh-save-github-token seems to be trying to use my active gptel-backend (Gemini) instead of the Copilot backend.

@marcus-lundgren
Copy link
Copy Markdown
Author

@karthink - Good catch! I've pushed a fix and I am now unable to replicate the issue.

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 6033888 to 97387ba Compare March 16, 2026 18:57
@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 97387ba to 4bb9d41 Compare March 22, 2026 21:04
@marcus-lundgren
Copy link
Copy Markdown
Author

I made an unfortunate rebase that was pushed. It seems like this PR was automatically closed due to that. Reopening it.

@karthink
Copy link
Copy Markdown
Owner

karthink commented Mar 24, 2026 via email

@marcus-lundgren
Copy link
Copy Markdown
Author

marcus-lundgren commented Mar 24, 2026

@karthink - Thank you for the update! It is always great to be humbled once in a while after having an accident like the one that we both seem to have shared. (Thank goodness for git reflog show!)

That sounds interesting. I may look into doing something with plstore instead of copying the secret manually from a password manager, as it has support for storing secrets (plstore-put). Will share a suitable snippet if I get around to it.

@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from 182b2b5 to ede3a7a Compare April 2, 2026 18:18
(float-time) will implicitly use (current-time) if the argument is not specified.
Removed the file based persistent storage of the chat token.

Introduced overloadable function variables that determines how a token
is loaded and saved.
This commit introduces proper support for more than one GitHub
account. By stating which account a backend should be considered to be
connected to, it is possible to ensure that we don't e.g. login using
a personal account to a backend considered for the work account.

The file based persistence will make use of the stated GitHub username
and make that a suffix to the configured file path. As the username
for GitHub accounts may only contain [0-9a-zA-Z], they can safely be
used for filenames.

The custom save/load function now takes the username as a parameter
so that it can decide on how to proceed.
@marcus-lundgren marcus-lundgren force-pushed the gh-change-token-management branch from ede3a7a to 4ade3d8 Compare April 4, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants