Is storing a reference to a (possibly) temporary object legal as long as the reference doesn't outlive the object?

After reading about std::variant and the handy overloaded trick I set myself a challenge (an exercise) to write a helper that would allow me to write

visit(variant).with([](int){}, [](char){});

instead of

std::visit(overloaded{[](int){}, [](char){}}, variant);

I'd also like it to work in other cases, like

VariantT variant;
VisitorT visitor;
visit(variant).with(visitor);
visit(VariantT{std::in_place_type<int>, 1}).with(visitor);
const VariantT constVariant;
visit(constVariant).with([](auto&&){});

You get the idea.

Here's what I came up with (note that I've changed the name from overloaded to Visitor:

#include <iostream>
#include <string>
#include <utility>
#include <variant>
template <typename... T>
struct Visitor : public T...
{
using T::operator()...;
};
template <typename... T>
Visitor(T...)->Visitor<T...>;
template <typename VariantT>
struct Helper
{
Helper(VariantT& variant)
: m_variant{variant}
{}
Helper(const Helper&) = delete;
Helper(Helper&&) = delete;
Helper& operator=(const Helper&) = delete;
Helper& operator=(Helper&&) = delete;
~Helper() = default;
template <typename VisitorT, typename... VisitorsT>
decltype(auto) with(VisitorT&& visitor,
VisitorsT&&... visitors) && // this function is ref-qualified so we can only call this on a temp object and we can
// be sure that the variant lives at least as long as us
{
if constexpr (sizeof...(visitors) == 0)
{
return std::visit(std::forward<VisitorT>(visitor), m_variant);
}
else
{
return std::visit(Visitor{visitor, visitors...}, m_variant);
}
}
private:
VariantT& m_variant;
};
template <typename VariantT>
decltype(auto) visit(VariantT&& variant)
{
// no forwarding here, even if an rvalue was passed, pass an lvalue ref
return Helper{variant};
}
int main()
{
visit(std::variant<int>{std::in_place_type<int>, -7})
.with([](int i) { std::cout << "got an int: " << i << std::endl; },
[](std::string str) { std::cout << "got a string: " << str << std::endl; });
std::variant<int, std::string> v = "String";
visit(v).with([](int i) { std::cout << "got an int: " << i << std::endl; },
[](std::string str) { std::cout << "got a string: " << str << std::endl; });
visit(v).with([](int& i) { i += 7; },
[](std::string& str) { str += "modified"; });
std::cout << visit(v).with([](int i) { return std::to_string(i); },
[](std::string str) { return str; }) << std::endl;
}

The question is: is storing references in the Helper like that perfectly legal?
My understanding is that temporary objects live until the end of the expression, so I guess this is ok?
Will it do The Right Thing? Are there any pitfalls in that code?

I've tested this code on both msvc and gcc and I'm not seeing anything wrong, but it doesn't mean this works fine in all cases.

The only obvious problem is that you can have things like

decltype(auto) silly() {
std::variant<int, std::string> v = "String";
return visit(v);
}
int main()
{
silly().with([](auto){ std::cout << "dangling" << std::endl; };
}

But such things are relatively easy to spot in review.