How to filter a dataframe with range having partitions as year, month, date and hour?

I have to read a dataframe from a table with partitions as follows - year, month, date and hour.

Input :

Need to filter a range of partitions from the table based on my audit range - start and end timestamps.

Start Timestamp : 2023-12-31 15:00:00 (Inclusive)
End Timestamp : 2024-01-01 14:00:00 (Exclusive)

Expected Output :

Tried below:

Try 1:

val filteredDf = rawDF.where(($"year" >= startTimeLocal.getYear && $"month" >= startTimeLocal.getMonthValue && $"day" >= startTimeLocal.getDayOfMonth && $"hour" >= startTimeLocal.getHour) && ($"year" <= endTimeLocal.getYear && $"month" <= endTimeLocal.getMonthValue && $"day" <= endTimeLocal.getDayOfMonth && $"hour" < endTimeLocal.getHour))

This condition fails as the hour values greater than 14 for 31st day would be skipped.

Try 2:

val yearDf = rawDF.where($"year" >= startTimeLocal.getYear && $"year" <= endTimeLocal.getYear)
val monthDf = yearDf.where(($"year" === startTimeLocal.getYear && $"month" >= startTimeLocal.getMonthValue) || ($"year" === endTimeLocal.getYear && $"month" <= endTimeLocal.getMonthValue))
val dayDf = monthDf.where(($"day" >= startTimeLocal.getDayOfMonth && $"hour" >= startTimeLocal.getHour) || ($"day" <= endTimeLocal.getDayOfMonth && $"hour" < endTimeLocal.getHour))

Try 3:

val final4Df = rawDF.where(($"year" >= startTimeLocal.getYear && $"day" >= startTimeLocal.getDayOfMonth && $"hour" >= startTimeLocal.getHour && $"day" >= startTimeLocal.getDayOfMonth) || ($"year" <= endTimeLocal.getYear && $"month" <= endTimeLocal.getMonthValue && $"day" <= endTimeLocal.getDayOfMonth && $"hour" < endTimeLocal.getHour))

I think you can start by including all data between the start and end years and then filter out on granular level

val startYear = startTimeLocal.getYear
val startMonth = startTimeLocal.getMonthValue
val startDay = startTimeLocal.getDayOfMonth
val startHour = startTimeLocal.getHour

val endYear = endTimeLocal.getYear
val endMonth = endTimeLocal.getMonthValue
val endDay = endTimeLocal.getDayOfMonth
val endHour = endTimeLocal.getHour

val filteredDf = rawDF.where(
    ($"year" >= startDay && $"year" <= endYear)
    && !(
        ($"year" === startYear && $"month" < startMonth) // All months before start date
        || ($"year" === startYear && $"month" === startMonth && $"day" < startDay) // All days before start day
        || ($"year" === startYear && $"month" === startMonth && $"day" === startDay && $"hour" < startHour) // All hours before start hour
        || ($"year" === endYear && $"month" > endMonth) // All months after end date
        || ($"year" === endYear && $"month" === endMonth && $"day" > endDay) // All days after end day
        || ($"year" === endYear && $"month" === endMonth && $"day" === endDay && $"hour" >= endHour) // All hours after end hour
    )
)