Django: Managing multi-value POST form fields with session data. List not passed after form validation redirect

I have an advanced search form in my Django app that I want to use the POST method with. Using GET reveals too much info about some inner workings of the app. Using POST requires some extra work to ensure pagination functions correctly.

I've found many great related answers:
https://stackoverflow.com/questions/2266554/paginating-the-results-of-a-django-forms-post-request?rq=4

I decided to store the POST query in session data to ensure it is available for pagination. I'm storing the entire POST as a QueryDict in the session as per:
https://www.abidibo.net/blog/2014/09/19/paginating-results-django-form-post-request/

For some reason after a successful validation of the form and subsequent redirect the session data loses any lists and retains just the last value in the list as a single item.

The simplified model (the actual model is much more complex):

class Organisation(models.Model):
    name = Charfield()
    location = Charfield()
    industry = Charfield() # This is a choices field with 19 options
...

class OrganisationSearchForm(forms.ModelForm):
    search_org = forms.CharField(label="Organisation")
    location = forms.Charfield(label="location")

    search_industry = forms.MultipleChoiceField(label="Industry", choices=[...#19 choices enumerated...])

I have two views with drive the search - a search form view and a search results view.

class OrganisationSearchFormView(FormView):

    class Meta:
        model = Organisation

    form_class = OrganisationSearchForm
    template_name = "organisation_search_form.html"
    success_url = reverse_lazy("organisation_search_results")

    def post(self, request):
        form = OrganisationSearchForm(request.POST)
        if form.is_valid():
            request.session['post-query'] = self.request.POST
            return self.form_valid(form)
        else:
            return self.form_invalid(form)    
...

class OrganisationSearchResultListView(ListView):
    model = Organisation
    template_name = "organisation_search_results.html"

    def get(self, request, *args, **kwargs):
        # if there is not post query in the session data, redirect user back to the search form
        if 'post-query' not in request.session:
            return HttpResponseRedirect(reverse_lazy("organisation_search"))
        return super().get(request, *args, **kwargs)

    def get_queryset(self, **kwargs):
        qs = super().get_queryset()
        post_query = QueryDict('').copy()
        post_query.update(self.request.session.get('post-query'))

        search_org = post_query.get('search_org')
        location = post_query.get('location')
        search_industry = post_query.getlist('search_industry') # .getlist is required for a multivalue object.

        qs = Organisation.objects.all()

        # I filter the queryset based on the query parameters

The workflow here is that the form view submits (POST method) to itself for validation. If the form is valid the QueryDict object is stored in the session data. Subsequently, there is an automatic redirect (GET method) to the Search Results View.

Form View -> Form View (Validation) -> Results View

During that redirect any lists within the QueryDict are stripped down to a single value. I've put a print statement in the form validation block of the form view post method and also in the get method of the results view which shows the change:

app-web-1  | [2023-07-17 15:31:03 +0000] [7] [DEBUG] POST /advsearch/i/
app-web-1  | <QueryDict: {'csrfmiddlewaretoken': ['************'], 'search_org': [''], 'search_industry': ['11', '21', '22', '23'], 'location': ['']}>
app-web-1  | [2023-07-17 15:31:04 +0000] [7] [DEBUG] GET /sr/i/
app-web-1  | <QueryDict: {'csrfmiddlewaretoken': ['************'], 'search_org': [''], 'search_industry': ['23'], 'location': ['']}>

I believe the issue is related to the use of a dict rather than QueryDict being used somewhere in that chain:
https://stackoverflow.com/questions/39565023/django-querydict-only-returns-the-last-value-of-a-list

If I change the form action to POST directly to the search results view, I do not have the same problem, the lists are retained and present in the session data. i.e.
Form View -> Results View

However, I think by posting directly to the results view that the form validation step is being missed. Is this correct?

What is the best approach to resolve this issue?
Is there a way to retain the list using the: Form View -> Form View (Validation) -> Results View workflow?

Any advice or guidance appreciated.

I came across a similar problem and worked out a solution that I think might also help in your case.

In summary:

• When you save request.POST to request.session, django serializes it and something about
  that process causes any fields in request.POST that contained multiple values to be
  truncated down to one value. Per your example, 'search_industry': ['11', '21', '22', '23']
  becomes 'search_industry': ['23'].
• To avoid losing data in that serialization process, you can write your own function to
  serialize any multi-value fields in request.POST before calling request.session.save().
• When getting the posted data out of the session in the next view, you would have to
  deserialize the values for any multi-value fields.
• Both the serialization and deserialization functions will need to know which keys in
  request.POST are associated with multi-value fields. For that, I added a hidden input
  to the form that lists all multiselect fields.

Here are my functions for serializing and deserializing multi-value fields in requset.POST:

def serialize_requestPOST_multivalue_items(requestPOST):  # pass in request.POST
    new_requestPOST = requestPOST.copy()  # a mutable QueryDict

    for key in requestPOST.getlist("multi_value_keys"):
        value = json.dumps(requestPOST.getlist(key))
        new_requestPOST.__setitem__(key, value)
    
    # The value of new_requestPOST["multi_value_keys"] also needs to be serialized.
    new_requestPOST.__setitem__(
        "multi_value_keys", json.dumps(requestPOST.getlist("multi_value_keys"))
    )

    return new_requestPOST


def deserialize_requestPOST_multivalue_items(requestPOST):
    requestPOST["multi_value_keys"] = json.loads(requestPOST["multi_value_keys"])

    for key in requestPOST["multi_value_keys"]:
        requestPOST[key] = json.loads(requestPOST[key])

    return requestPOST

Here's a mixin to add to forms when you'll want the option of saving request.POST to request.session:

class SerializerPrereqMixin:
    def add_multi_values_hidden_field(self):
        multi_value_keys = [
            field_key for field_key, field in self.fields.items() 
            if isinstance(
                field.widget, (SelectMultiple, CheckboxSelectMultiple)
            )
        ]
        self.fields["multi_value_keys"] = CharField(
            widget=MultipleHiddenInput(),
            required=False,
            initial=multi_value_keys,
        )

With this, in the first view you would so something like:

requestPOST = serialize_requestPOST_multivalue_items(request.POST)
request.session["posted_data"] = requestPOST
request.session.save()

And in the view where you want to used the posted data:

posted_data = request.session.get("posted_data")
posted_data = deserialize_requestPOST_multivalue_items("posted_data")