ASGI headers presented as str instead of bytes

[jessestimpson]

Hi there, thanks for the useful projects.

While I was toying around with hornbeam, I was getting errors on authorization for a python webapp that I was running. I'm way out of my depth here w.r.t the python API contracts, so I was using Claude.

The eventual solution, according to the LLM, was to normalize ASGI headers as bytes instead of str. It did work and it sounds plausible, so I thought it'd be best to present it as an issue here.

Specifically, the diff it suggested is

--- a/c_src/py_asgi.c
+++ b/c_src/py_asgi.c
@@ -1096,6 +1096,38 @@ static PyObject *asgi_scope_from_map(ErlNifEnv *env, ERL_NIF_TERM scope_map) {
         } else if (py_key == ASGI_KEY_QUERY_STRING || py_key == ASGI_KEY_RAW_PATH) {
             ErlNifBinary bin;
             if (enif_inspect_binary(env, value, &bin) && bin.size == 0) {
                 Py_INCREF(ASGI_EMPTY_BYTES);
                 py_value = ASGI_EMPTY_BYTES;
             }
+        } else if (py_key == ASGI_KEY_HEADERS) {
+            /* ASGI spec: headers must be list[tuple[bytes, bytes]].
+             * Convert each [[name_bin, value_bin], ...] element to [bytes, bytes]. */
+            unsigned int list_len;
+            if (enif_get_list_length(env, value, &list_len)) {
+                py_value = PyList_New(list_len);
+                if (py_value == NULL) goto error;
+                ERL_NIF_TERM head, tail = value;
+                Py_ssize_t idx = 0;
+                while (enif_get_list_cell(env, tail, &head, &tail)) {
+                    /* Each element is a 2-element list [name, value] */
+                    ERL_NIF_TERM hname_term, hvalue_term, htail;
+                    if (!enif_get_list_cell(env, head, &hname_term, &htail) ||
+                        !enif_get_list_cell(env, htail, &hvalue_term, &htail)) {
+                        Py_DECREF(py_value);
+                        py_value = NULL;
+                        break;
+                    }
+                    ErlNifBinary name_bin, value_bin;
+                    if (!enif_inspect_binary(env, hname_term, &name_bin) ||
+                        !enif_inspect_binary(env, hvalue_term, &value_bin)) {
+                        Py_DECREF(py_value);
+                        py_value = NULL;
+                        break;
+                    }
+                    PyObject *pair = PyList_New(2);
+                    if (pair == NULL) { Py_DECREF(py_value); py_value = NULL; break; }
+                    PyList_SET_ITEM(pair, 0, PyBytes_FromStringAndSize((char *)name_bin.data, name_bin.size));
+                    PyList_SET_ITEM(pair, 1, PyBytes_FromStringAndSize((char *)value_bin.data, value_bin.size));
+                    PyList_SET_ITEM(py_value, idx++, pair);
+                }
+            }
         }
 
         /* Generic conversion if no optimization applied */

py_asgi.c's asgi_scope_from_map iterates the Erlang scope map and handles well-known keys like type, method, path with optimised conversions. But headers falls through to the generic term_to_py fallback, which converts Erlang binaries (<<"content-type">>) to Python str. The ASGI spec mandates list[tuple[bytes, bytes]] for headers. Starlette searches for b"content-type" (bytes) when parsing form bodies and finds nothing — so Form(...) parameters resolve to empty strings and any auth attempt fails with 401, silently.

If this is nonsense, I apologize and please feel free to close and ignore. I really have no clue! Anyway, thanks again.