How to apply affine transform to part of the image?

There is an image with 3 anchor points (black rectangle). An affine transformation can be generated from these points.

It's easy to transform the whole image with the generated matrix, but if I want to transform a part of the image (green rectangle), how to do it?

The only way I know is to extend the green rectangle to full size image, transform, then crop it again.

Update to make it more clear.
If I have the image above and a transformer, the transformed image is like:

Because the image is big and takes long time to complete the transformation, so I only want to transform part of the image (the green rectangle)

what I expect is crop the green rectangle and perform some transformation to it, to get the same result.

Basically we may chain the transformation matrix of a "ROI sized image" with translation matrix that "shifts" the ROI to the top-left corner of the "large image".

Assume we have an image in the size of the ROI (name it roi):

Assume we know the transformation matrix M for that given ROI image:

<!-- language: lang-python -->

M = np.array([[ 1.03366188, 0.37622216,  50],
              [-0.37622216, 1.03366188, 100]])

Here is an example for computing the transformation matrix M:

<!-- language: lang-python -->

import cv2
import numpy as np

img = cv2.imread('input_image.png');

x0, y0 = 770, 491  # Top left corner of the ROI in the image.
roi = img[y0:685, x0:1129, :]
M = cv2.getRotationMatrix2D((roi.shape[1]/2, roi.shape[0]/2), 20, 1.1)  # Rotation and scale
M[0:2, 2] = (50, 100)  # Add translation
print(M)

Building transformation matrix be applied on the "large image" with the given ROI:

• Prepare transformation matrix that applies translation.
  The translation "shifts" each pixel in the "large image" to bring the ROI to the top-left corner of the image.
  Translation matrix:

<!-- language: lang-python -->

    T = np.array([[1, 0, -x0],
                  [0, 1, -y0],
                  [0, 0,  1]])

• Add row [0, 0, 1] to the bottom of M.
  OpenCV convention for affine transformation is omitting the bottom row that equals [0, 0, 1].
  We have to add the omitted row for making M size 3x3.

<!-- language: lang-python -->

    M = np.vstack((M, np.array([0, 0, 1])))

• Chain transformation - multiply M by the translation matrix T:

<!-- language: lang-python -->

    roiM = M @ T

• Remove the last row of roiM for matching OpenCV 2x3 affine transformation conventions:

<!-- language: lang-python -->

    roiM = roiM[0:2, :]

• Apply warpAffine to the "large image" with roiM transformation matrix:

<!-- language: lang-python -->

    rotated_roi = cv2.warpAffine(img, roiM, (dst_width, dst_height))

For testing, we may apply warpAffine to roi with matrix M:

<!-- language: lang-python -->

rotated_roi_ref = cv2.warpAffine(roi, M, (dst_width, dst_height))

Code sample:

<!-- language: lang-python -->

import cv2
import numpy as np

img = cv2.imread('input_image.png');

x0, y0 = 770, 491    # Top left corner of the ROI in the image.
#roi = img[y0:685, x0:1129, :]
#M = cv2.getRotationMatrix2D((roi.shape[1]/2, roi.shape[0]/2), 20, 1.1)  # Rotation and scale
#M[0:2, 2] = (50, 100)  # Add translation
#print(M)

M = np.array([[ 1.03366188, 0.37622216,  50],
              [-0.37622216, 1.03366188, 100]])

dst_width = 538  # int(roi.shape[1]*1.5)
dst_height = 291  # int(roi.shape[0]*1.5)

# Translation matrix - "shifts" the ROI to the top-left corner of img
T = np.array([[1, 0, -x0],
              [0, 1, -y0],
              [0, 0,  1]])

M = np.vstack((M, np.array([0, 0, 1])))  # Add row [0, 0, 1] to the bottom of M (we need M to be 3x3 before matrix multiplication).

roiM = M @ T  # Chain transformations - each source pixel is translated by x=-770 and y=-491 and warped with M after the translation.
roiM = roiM[0:2, :]  # Keep only the two upper rows (warpAffine input is 2x3 matrix).

rotated_roi = cv2.warpAffine(img, roiM, (dst_width, dst_height))


# Reference for testing: apply warp to roi
################################################################################
roi = img[y0:685, x0:1129, :]
M = cv2.getRotationMatrix2D((roi.shape[1]/2, roi.shape[0]/2), 20, 1.1)  # Rotation and scale
M[0:2, 2] = (50, 100)  # Add translation
rotated_roi_ref = cv2.warpAffine(roi, M, (dst_width, dst_height))
################################################################################


# Show output images for testing:
#cv2.imshow('roi', roi)
cv2.imshow('rotated_roi', rotated_roi)
cv2.imshow('rotated_roi_ref', rotated_roi_ref)
cv2.waitKey()
cv2.destroyAllWindows()

rotated_roi:

rotated_roi_ref (reference for testing):

Update:

Different assumptions:

• Assume the transformation matrix of the "large image" is given.
• Assume we want the center of the ROI is the "large image" to be transformed to the center of the desired destination image.

Code sample (please read the comments):

<!-- language: lang-python -->

import cv2
import numpy as np

img = cv2.imread('input_image.png');

x0, y0 = 770, 491    # Top left corner of the ROI in the image.
x1, y1 = 1128, 684   # Bottom left corner of the ROI

# Size of destination image (output of Warp)
dst_width = 538
dst_height = 291

# Assume the following transformation matrix M is given:
M = cv2.getRotationMatrix2D((img.shape[1]/2, img.shape[0]/2), 15, 1.05)  # Transformation matrix that applies the "large image"

# Center coordinate of the ROI:
roi_cx = (x1 + x0) / 2
roi_cy = (y1 + y0) / 2

# Transform the center coordinate of the ROI and get the center coordinate after the transformation (after the transformation of the "large image").
transformed_roi_cx, transformed_roi_cy = cv2.transform(np.array([[[roi_cx, roi_cy]]], np.float64), M)[0][0]

# Center coordinate of the destination image (output of Warp).
dst_cx = (dst_width - 1) / 2
dst_cy = (dst_height - 1) / 2

# Compute the horizontal and vertical translation - how much to shift the center after the "original warp".
tx = dst_cx - transformed_roi_cx
ty = dst_cy - transformed_roi_cy

# Translation matrix:
T = np.array([[1, 0, tx],
              [0, 1, ty],
              [0, 0, 1]])

M = np.vstack((M, np.array([0, 0, 1])))  # Add row [0, 0, 1] to the bottom of M (we need M to be 3x3 before matrix multiplication).

# T is multiplied from the left side - assume warp with M and than "shift" (translation comes after the warp).
roiM = T @ M  # Chain transformations
roiM = roiM[0:2, :]  # Keep only the two upper rows (warpAffine input is 2x3 matrix).

rotated_roi = cv2.warpAffine(img, roiM, (dst_width, dst_height))

cv2.imwrite('rotated_roi.png', rotated_roi)  # Save for testing.

rotated_roi:

Create an image that extracts only the interest rectangular region.

The transformation between pixel positions in this ROI image and pixel positions in the original image is a simple translation.

So, add the translation for this coordinate transformation to the affine transformation and apply it to the ROI image.
Here, the size of the transformed image buffer is the size of the original image.

e.g. consider about the warp from original point to result point.

Result_Position = AffineTrans( Original_Point )

If ROI is (Left=100,Top=200, Width=any,Height=any) rectangle in the original image,
Point(0,0) in the extracted ROI image corresponds to (100,200) in the original image.

Since the argument of the function AffineTrans() is "Point in the Original Image", to use this function for the ROI image, you have to obtain ther argument value by translating the position in the ROI image to in the Original image.
i.e.

Result_Position = AffineTrans( Point_in_ROI_Image + (100,200) )

(Of cause, in real, necessary warp direction is the opposite, but what to do is same.)