Why only Python 3.8 - Azure ML kernel can find openai module in notebook

In Azure ML Studio, I created a notebook, installed some packages and tried to run a code

!pip install -r requirements.txt

above worked

! pip show openai

Result of pip show openai

Name: openai
Version: 0.25.0
Summary: Python client library for the OpenAI API
Home-page: https://github.com/openai/openai-python
Author: OpenAI
Author-email: support@openai.com
License: None
Location: /anaconda/envs/azureml_py38/lib/python3.8/site-packages
Requires: typing-extensions, pandas, requests, openpyxl, pandas-stubs, tqdm, numpy
Required-by: 

Code

import openai
import re
import requests
import sys
from num2words import num2words
import os
import pandas as pd
import numpy as np
from openai.embeddings_utils import get_embedding, cosine_similarity
from transformers import GPT2TokenizerFast

#API_KEY = os.getenv("AZURE_OPENAI_API_KEY") 
API_KEY = "somekey"
#RESOURCE_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT") 
RESOURCE_ENDPOINT = "https://someendpoint/"
openai.api_type = "azure"
openai.api_key = API_KEY
openai.api_base = RESOURCE_ENDPOINT
openai.api_version = "2022-12-01"

url = openai.api_base + "/openai/deployments?api-version=2022-12-01"

r = requests.get(url, headers={"api-key": API_KEY})

print(r.text)

I found that only Python 3.8 Azure ML could find openai module. When I selected some other kernel then I got error moduel openai not found. Why?

Error when other kernel is selected

ModuleNotFoundError                       Traceback (most recent call last)
Cell In[1], line 1
----> 1 import openai
      2 import re
      3 import requests

ModuleNotFoundError: No module named 'openai'

Just make sure to use the right prefix for the pip command.
In my case I ran '%pip' and not '!pip' when installing openai.

With '%pip' the packages are installed in the same environment as you notebook and not in the shell

I tried in my environment and got the below results:

Initailly, I got the same error in my environment.

Error:

   ModuleNotFound Error
    Input In [9], in <cell line: 1>()
    ----> 1 import openai
    2 import re
    3 import requests
    Traceback (most recent call last)
    ModuleNotFound Error:No module named openai '

The reason why the openai module is not found when you switch to a different kernel in Azure ML Studio is because the module is installed in the Python environment associated with the notebook's kernel. Each kernel in Azure ML Studio has its own isolated Python environment, which means that the packages you install in one kernel will not be available in another kernel.

You need directly install by commands.when I tried with same code and packages it executed successfully:

!pip install openai==0.27.4
!pip install pandas==2.0.0
!pip install num2words==0.5.12

Code:

import openai
import re
import requests
import sys
from num2words import num2words
import os
import pandas as pd
import numpy as np
from openai.embeddings_utils import get_embedding, cosine_similarity
from transformers import GPT2TokenizerFast

#API_KEY = os.getenv("AZURE_OPENAI_API_KEY") 
API_KEY = "somekey"
#RESOURCE_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT") 
RESOURCE_ENDPOINT = "https://someendpoint/"
openai.api_type = "azure"
openai.api_key = API_KEY
openai.api_base = RESOURCE_ENDPOINT
openai.api_version = "2022-12-01"

url = openai.api_base + "/openai/deployments?api-version=2022-12-01"

r = requests.get(url, headers={"api-key": API_KEY})

print(r.text)

Output:

{
  "data": [
    {
      "scale_settings": {
        "scale_type": "standard"
      },
      "model": "gpt-35-turbo",
      "owner": "organization-owner",
      "id": "deploymentname1",
      "status": "succeeded",
      "created_at": 16807xxx,
      "updated_at": 16807xxx,
      "object": "deployment"
    }
  ],
  "object": "list"
}

Reference:
Run Jupyter notebooks in your workspace - Azure Machine Learning | Microsoft Learn