Misaligned nested tabulars inside another tabular

I want to make a table where the hlines start to the second column. In order to do so, I nested tabulars but they don't align well as you can see :

Is it possible to solve this alignment problem with nested tabulars ? I'd prefer not to use extra libraries unless the code becomes awful to manage.

I'm open to other solutions without extra libraries of course. For example, I tried using a single tabular but the hlines extend to the first column which is not what I want :

It might be a simple solution I don't know for this without using multirow.

Here is a minimal reproducible example for both tabular and nested tabulars that is not aligned :

\documentclass[12pt]{standalone}
\begin{document}
\begin{tabular}{l|ccccc||c|}
	\hline
	$t_6$	&	0	&	0	&	0	&	-1	&	0	&	0
	\\
	\hline
	$t_5$	&	0	&	0	&	0	&	0	&	-1	&	0
	\\
	\hline
	$t_4$	&	0	&	0	&	0	&	0	&	0	&	0
	\\
	\hline
	$t_3$	&	0	&	0	&	0	&	0	&	1	&	0
	\\
	\hline
	$t_2$	&	0	&	0	&	0	&	1	&	0	&	0
	\\
	\hline
	$t_1$	&	0	&	0	&	1	&	0	&	0	&	0
	\\
	\hline
	$t_0$	&	0	&	1	&	0	&	0	&	0	&	0
	\\
	\hline
\end{tabular}
\end{document}

\documentclass[12pt]{standalone}
\begin{document}
\begin{tabular}{l cccccc}
	$t_6$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	0	&	-1	&	0	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_5$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	0	&	0	&	-1	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_4$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	0	&	0	&	0	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_3$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	0	&	0	&	1	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_2$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	0	&	1	&	0	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_1$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	0	&	1	&	0	&	0	&	0
	\\
	\hline
	\end{tabular}
	\\
	$t_0$	&
	\begin{tabular}{|ccccc||c|}
	\hline
	0	&	1	&	0	&	0	&	0	&	0
	\\
	\hline
	\end{tabular}
\end{tabular}
\end{document}

If it is not possible or way too hard to solve this problem and obtain a well aligned table like the second one (who is well aligned only because every cells have the same amount of characters) without extra library I won't insist. I just think it is good practice to avoid loading libraries if I can.

Between the lines I read that you want things to be easy (don't we all?) because you don't want to include any additional packages. This is really not how things work, in general. The basic classes in LaTeX are extended by the use of packages - don't let that be a concern; in many cases they make things simpler to use.

Also, your alignment issue assessment is correct, but can be corrected using some basic tabular constructions.

Below I've added the array package - part of every TeX distribution - which allows for some extended column specifications. In particular, w{<align>}{<width>} that sets the <align>ment within a fixed-<width> column. For example, wc{2em} will center its contents within a 2em-width column. The alternative is to use >{\centering\arraybackslash}p{2em}, but this also requires array. If you really don't want to use array, you can just use p{2em} and then issue \centering in every cell... but such replication makes the code harder to maintain.

\documentclass{article}

\usepackage{array}

\newlength{\cellwidth}
\setlength{\cellwidth}{1em}
\NewDocumentCommand{\myvector}{ m O{5} m }{%
  \makebox[2em][l]{#1}%
  $\begin{array}{ | *{#2}{wc{\cellwidth}} || c | }
    \hline
    #3 \\
    \hline
  \end{array}$%
}

\begin{document}

\begin{tabular}{l}
  \myvector{$t_6$}{0 & 0 & 0 & -1 &  0 & 0} \\
  \myvector{$t_5$}{0 & 0 & 0 &  0 & -1 & 0} \\
  \myvector{$t_4$}{0 & 0 & 0 &  0 &  0 & 0} \\
  \myvector{$t_3$}{0 & 0 & 0 &  0 &  1 & 0} \\
  \myvector{$t_2$}{0 & 0 & 0 &  1 &  0 & 0} \\
  \myvector{$t_1$}{0 & 0 & 1 &  0 &  0 & 0} \\
  \myvector{$t_0$}{0 & 1 & 0 &  0 &  0 & 0}
\end{tabular}

\end{document}

What you'll see in the above code is a macro called \myvector{<label>}[<# elements>]{<elements>} that takes 2 mandatory arguments: <label> set to the left of the vector in a left-aligned fixed width box of width 2em, and the <elements> of that vector using the regular tabular-like specification (element & element & ...). The second argument <# elements> is optional, and defaults to 5 - the number of elements before the final element/column. If you want a larger/smaller vector, you can use that argument to adjust the display, as in

\myvector{X}[3]{1 & 2 & 3 & 4}

or

\myvector{Y}[7]{1 & 2 & 3 & 4 & 5 & 6 & 7 & 8}