Aligning multiple texts center if one is null

I want to align two texts to be centered vertically to the start of the TaskCard. It works as intended if there are two texts present. However, if there is only one and the other one is null the null one will completely disappear, but will still leave a blank space in its place. How do I get rid of it so I can properly align the text?

@Composable
fun TaskCard(
    modifier: Modifier = Modifier,
    task: Task,
    viewModel: TaskListViewModel = hiltViewModel()
) {
    Card(
        colors = CardDefaults.cardColors(
            containerColor = MaterialTheme.colorScheme.surfaceVariant
        ), modifier = modifier.padding()
    ) {
        Surface(
            color = MaterialTheme.colorScheme.surfaceVariant, modifier = modifier
        ) {
            Row(
                verticalAlignment = Alignment.CenterVertically, modifier = modifier.padding(12.dp)

            ) {
                Column(
                    modifier = modifier
                        .weight(1f)
                        .padding(12.dp)
                ) {
                    task.categoryName?.let {
                        Text(
                            text = it,
                            style = MaterialTheme.typography.titleMedium,
                            color = MaterialTheme.colorScheme.primary
                        )
                    }
                    Text(
                        text = task.taskName,
                        style = MaterialTheme.typography.headlineSmall,
                        color = MaterialTheme.colorScheme.onBackground,
                    )
                }
                IconButton(
                    onClick = { viewModel.onEvent(TaskListEvent.OnEditTask(task)) },
                    modifier = modifier.size(36.dp)
                ) {
                    Icon(
                        imageVector = Icons.Rounded.Edit,
                        contentDescription = "Edit",
                        modifier = modifier.padding(horizontal = 2.dp),
                        tint = MaterialTheme.colorScheme.onBackground
                    )
                }
                IconButton(
                    onClick = { }, modifier = modifier.size(36.dp)
                ) {
                    Icon(
                        imageVector = Icons.Rounded.Notifications,
                        contentDescription = "Notifications",
                        modifier = modifier.padding(horizontal = 2.dp),
                        tint = MaterialTheme.colorScheme.onBackground
                    )
                }
            }
        }
    }
}

As Gabriele Mariotti suggested I've changed the condition under which the code is executed.

from:

task.categoryName?.let

to:

if (!task.categoryName.isNullOrEmpty())