如何在Ubuntu 20.04上构建并安装PyLucene。

huangapple
go评论122阅读模式
英文:

How to build and install pylucene on ubuntu 20.04

问题

我正试图在我安装了干净的 WSL Ubuntu 20.04 上安装 Pylucene。我尝试按照官方页面上的教程，但看起来已经过时。因此，我想知道这里是否有人成功在 Ubuntu 20.04 和 Python 3.8.2 上使其工作。

我运行的命令：

  1. sudo apt-get upgrade
  2. sudo apt-get install -y default-jdk ant build-essential python3-dev
  3. mkdir pylucene
  4. cd pylucene
  5. curl https://downloads.apache.org/lucene/pylucene/pylucene-8.3.0-src.tar.gz | tar -xz --strip-components=1
  6. cd jcc
  7. export JCC_JDK=/usr/lib/jvm/default-java
  8. python3 setup.py build

^^^^^
在这里失败：

  1. ...
  2. building 'jcc3' extension
  3. x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/jcc.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
  4. x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/JCCEnv.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
  5. x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -o build/lib.linux-x86_64-3.8/libjcc3.so -L/usr/lib/jvm/default-java/jre/lib/amd64 -ljava -L/usr/lib/jvm/default-java/jre/lib/amd64/server -ljvm -Wl,-rpath=/usr/lib/jvm/default-java/jre/lib/amd64:/usr/lib/jvm/default-java/jre/lib/amd64/server -Wl,-S
  6. /usr/bin/ld: 无法找到 -ljava
  7. /usr/bin/ld: 无法找到 -ljvm
  8. collect2: 错误：ld 返回 1 退出状态
  9. 错误：命令 'x86_64-linux-gnu-g++' 失败，退出状态 1

接下来我计划运行的命令：

  1. sudo python3 --preserve-env=JCC_JDK setup.py install
  2. cd ..
  3. make
  4. make test
  5. sudo make install
英文:

I am trying to install Pylucene on my WSL Ubuntu 20.04 clean installation. I tried to follow tutorial on the official page but it looks outdated. So I was wondering if anyone here managed to make it work on Ubuntu 20.04 and python 3.8.2

The commands I run:

  1. sudo apt-get upgrade
  2. sudo apt-get install -y default-jdk ant build-essential python3-dev
  3. mkdir pylucene
  4. cd pylucene
  5. curl https://downloads.apache.org/lucene/pylucene/pylucene-8.3.0-src.tar.gz | tar -xz --strip-components=1
  6. cd jcc
  7. export JCC_JDK=/usr/lib/jvm/default-java
  8. python3 setup.py build

^^^^^
Fails here on:

  1. ...
  2. building 'jcc3' extension
  3. x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/jcc.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
  4. x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_jcc_lib -DJCC_VER="3.7" -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux -I_jcc3 -Ijcc3/sources -I/usr/include/python3.8 -c jcc3/sources/JCCEnv.cpp -o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
  5. x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/jcc3/sources/jcc.o build/temp.linux-x86_64-3.8/jcc3/sources/JCCEnv.o -o build/lib.linux-x86_64-3.8/libjcc3.so -L/usr/lib/jvm/default-java/jre/lib/amd64 -ljava -L/usr/lib/jvm/default-java/jre/lib/amd64/server -ljvm -Wl,-rpath=/usr/lib/jvm/default-java/jre/lib/amd64:/usr/lib/jvm/default-java/jre/lib/amd64/server -Wl,-S
  6. /usr/bin/ld: cannot find -ljava
  7. /usr/bin/ld: cannot find -ljvm
  8. collect2: error: ld returned 1 exit status
  9. error: command 'x86_64-linux-gnu-g++' failed with exit status 1

Commands I plan to run afterwards:

  1. sudo python3 --preserve-env=JCC_JDK setup.py install
  2. cd ..
  3. make
  4. make test
  5. sudo make install

答案1

得分: 1

以下是在Ubuntu 18.04上成功安装Pylucene的步骤 - 这可能适用于您：

  1. 安装openjdk-8：

    1. apt install openjdk-8-jre openjdk-8-jdk openjdk-8-doc

    确保您已安装ant，如果没有，请运行 apt install ant。请注意，如果您安装了不同版本的openjdk，您需要将其删除或运行 update-alternatives 以使用版本1.8.0。

  2. 使用 java -version 检查Java版本是否为1.8.0*。

  3. 在安装openjdk-8之后创建符号链接（稍后会用到）：

    1. cd /usr/lib/jvm
    2. ln -s java-8-openjdk-amd64 java-8-oracle

  4. 安装python-dev：sudo apt install python-dev
    在我的情况下，Python 3不起作用，所以我最终使用了Python 2。但这可能不是问题的实际原因，所以您可以尝试使用Python 3。如果您选择使用Python 3，在下面的命令中将 python 替换为 python3。

  5. 安装JCC（在您的pylucene文件夹的jcc子文件夹中）：

    1. python setup.py build
    2. python setup.py install

    您在步骤3中创建的符号链接将在这里有帮助，因为此路径已硬编码到setup.py中 - 您可以进行检查。

  6. 安装pylucene（从您的pylucene文件夹的根目录）。
    编辑Makefile，根据您的设置取消注释/编辑变量。
    在我的情况下，它是这样的：

    1. PREFIX_PYTHON=/usr
    2. ANT=ant
    3. PYTHON=$(PREFIX_PYTHON)/bin/python
    4. JCC=$(PYTHON) -m jcc --shared
    5. NUM_FILES=10

    然后运行：

    1. make
    2. make test
    3. sudo make install

  7. 如果您看到与JCC的共享模式相关的错误，请从Makefile中删除 --shared。

英文:

Here are the steps that lead to the successful installation of pylucene on Ubuntu 18.04 - this may work for you:

  1. Install openjdk-8:
    apt install openjdk-8-jre openjdk-8-jdk openjdk-8-doc
    Ensure that you have ant installed, if you don't run apt install ant. Note that if you had a different version of openjdk installed you need to either remove it or run update-alternatives so that version 1.8.0 is used.

  2. Check that Java version is 1.8.0* with java -version

  3. After installing openjdk-8 create a symlink (you'll need it later):

  1. cd /usr/lib/jvm
  2. ln -s java-8-openjdk-amd64 java-8-oracle

  1. Install python-dev: sudo apt install python-dev
    In my case Python 3 didn't work so I ended up using Python 2. But this might not have been the actual reason of the problem, so you're welcome to try Python 3. If you go with Python 3, use python3 instead of python in the commands below.

  2. Install JCC (in jcc subfolder of your pylucene folder):

  1. python setup.py build
  2. python setup.py install

The symlink you created on step 3 will help here because this path is hardcoded into setup.py - you can check that.

  1. Install pylucene (from the root of your pylucene folder).
    Edit Makefile, uncomment/edit the variables according to your setup.
    In my case it was
  1. PREFIX_PYTHON=/usr
  2. ANT=ant
  3. PYTHON=$(PREFIX_PYTHON)/bin/python
  4. JCC=$(PYTHON) -m jcc --shared
  5. NUM_FILES=10

Then run

  1. make
  2. make test
  3. sudo make install
  1. If you see an error related to the shared mode of JCC remove --shared from Makefile.

答案2

得分: 0

已确认由@code-your-dream发布的答案在Python 3下也有效（尤其是在Ubuntu 18.04.1中）。

对我来说，通过这种方式安装jcc非常重要。我尝试通过conda安装，但在制作pylucene时出现了冲突。

在我这种情况下，还需要修改jcc的setup.py文件，将
'linux': ['-fno-strict-aliasing', '-Wno-write-strings']，
改为
'linux': ['-fno-strict-aliasing', '-Wno-write-strings'，'-D__STDC_FORMAT_MACROS']，
正如这个其他主题中提到的 https://stackoverflow.com/questions/46895399/issue-with-installing-pylucene-6-5-0-on-linux

要确认所需的Java版本（在我的情况下也是8，而不是11），您可以在pylucene文件夹的/jcc/setup.py中查找JDK = {...}块。在我的情况下，参考如下：
'linux': '/usr/lib/jvm/java-8-oracle'，

英文:

Confirmed that the answer posted by @code-your-dream works also with Python3 (and in particular in a Ubuntu 18.04.1)

For me was important to install jcc that way. I tried installing via conda and there were conflicts in the make of pylucene.

In my case also was needed to modify the setup.py file from jcc changing
'linux': ['-fno-strict-aliasing', '-Wno-write-strings'],
by
'linux': ['-fno-strict-aliasing', '-Wno-write-strings','-D__STDC_FORMAT_MACROS'],
as mentioned in this other thread https://stackoverflow.com/questions/46895399/issue-with-installing-pylucene-6-5-0-on-linux

To confirm which is the version of java needed (in my case also 8, didn't work with 11), you can search in the pylucene folder /jcc/setup.py the block JDK = {...}. In my case with the reference:
'linux': '/usr/lib/jvm/java-8-oracle',

答案3

得分: 0

  1. # Dockerfile for PyLucene 8.11.0, Python 3.9, JDK11, and Ubuntu 20.04 (focal)
  3. FROM ubuntu:focal
  5. ARG PYTHON_VERSION=3.9
  6. ARG PYLUCENE_VERSION=8.11.0
  8. ENV LANG=C.UTF-8
  10. RUN apt-get update && \
  11.     apt-get upgrade -y && \
  12.     apt-get install -y --no-install-recommends \
  13.         software-properties-common gpg-agent && \
  14.     add-apt-repository ppa:deadsnakes/ppa && \
  15.     apt-get update && \
  16.     apt-get remove -y software-properties-common && \
  17.     apt-get purge --auto-remove -y && \
  18.     apt-get clean
  20. RUN which gpg-agent
  22. RUN apt-get install -y --no-install-recommends \
  23.         "python$PYTHON_VERSION-dev" \
  24.         python3-setuptools \
  25.         python3-pip && \
  26.     apt-get remove -y gpg-agent && \
  27.     apt-get purge --auto-remove -y && \
  28.     apt-get clean
  30. RUN apt-get install -y --no-install-recommends \
  31.     build-essential \
  32.     ant \
  33.     jcc \
  34.     curl \
  35.     git \
  36.     default-jdk 
  38. RUN ls /usr/bin/ | grep "python"
  39. RUN ln -s $(which python3.9) /usr/bin/python
  41. RUN which python3.9 && which python && python --version
  43. WORKDIR /usr/lib/jvm/default-java/jre/lib
  44. RUN ln -s ../../lib amd64
  46. RUN java --version && javac --version
  48. RUN apt-get install -y --no-install-recommends \
  49.     libffi-dev \
  50.     zlib1g-dev
  52. WORKDIR /usr/src/pylucene
  53. RUN curl https://dlcdn.apache.org/lucene/pylucene/pylucene-$PYLUCENE_VERSION-src.tar.gz | tar -xz
  55. ENV PREFIX_PYTHON=/usr \
  56.     JCC_JDK=/usr/lib/jvm/default-java \
  57.     ANT=ant \
  58.     JCC='python -m jcc' \
  59.     NUM_FILES=10 \
  60.     PYTHON=python \
  61.     NO_SHARED=1
  63. RUN cd "pylucene-$PYLUCENE_VERSION/lucene-java-$PYLUCENE_VERSION/lucene" && \
  64.     ant ivy-bootstrap && \
  65.     ant && \
  66.     cd ../../../
  68. RUN cd "pylucene-$PYLUCENE_VERSION/jcc" && \
  69.     ls -la && \
  70.     NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py build && \
  71.     NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py install && \
  72.     cd .. && \
  73.     make JCC='python -m jcc' ANT=ant PYTHON=python NUM_FILES=8 && \
  74.     make install JCC='python -m jcc' ANT=ant PYTHON=python NUM_FILES=8 && \
  75.     cd ../../
  77. RUN apt-get remove -y gpg-agent ant jcc build-essential && \
  78.     apt-get purge --auto-remove -y && \
  79.     apt-get clean
  81. WORKDIR /usr/src
  82. RUN rm -rf pylucene
  84. RUN python -c "import lucene; lucene.initVM()"
  86. WORKDIR /app
  87. COPY . .
英文:

I wrote this dockerfile for PyLucene 8.11.0, Python 3.9, JDK11 (default-jdk) and ubuntu 20.04 (focal). Do extend with your favorite python package manager such as Poetry, Pipenv or Conda etc.

Dockerfile

  1. FROM ubuntu:focal
  2. ARG PYTHON_VERSION=3.9
  3. ARG PYLUCENE_VERSION=8.11.0
  4. # Uncomment to install specific version of poetry
  5. ENV LANG=C.UTF-8
  6. # ADD Python PPA Repository
  7. RUN apt-get update && \
  8. apt-get upgrade -y && \
  9. apt-get install -y --no-install-recommends \
  10. software-properties-common gpg-agent && \
  11. add-apt-repository ppa:deadsnakes/ppa && \
  12. apt-get update && \
  13. apt-get remove -y software-properties-common && \
  14. apt-get purge --auto-remove -y && \
  15. apt-get clean
  16. RUN which gpg-agent
  17. # Install Python
  18. RUN apt-get install -y --no-install-recommends \
  19. "python$PYTHON_VERSION-dev" \
  20. python3-setuptools \
  21. python3-pip && \
  22. apt-get remove -y gpg-agent && \
  23. apt-get purge --auto-remove -y && \
  24. apt-get clean
  25. # ======================== START OF ADDITIONAL INSTALLATION ========================
  26. # Install Java
  27. RUN apt-get install -y --no-install-recommends \
  28. build-essential \
  29. ant \
  30. jcc \
  31. curl \
  32. git \
  33. default-jdk 
  34. RUN ls /usr/bin/ | grep "python"
  35. RUN ln -s $(which python3.9) /usr/bin/python
  36. RUN which python3.9 && which python && python --version
  37. WORKDIR /usr/lib/jvm/default-java/jre/lib
  38. RUN ln -s ../../lib amd64
  39. # Java 11
  40. RUN java --version && javac --version
  41. # Installing PyLucene
  42. RUN which ant && ant -version
  43. RUN apt-get install -y --no-install-recommends \
  44. libffi-dev \
  45. zlib1g-dev
  46. WORKDIR /usr/src/pylucene
  47. RUN curl https://dlcdn.apache.org/lucene/pylucene/pylucene-$PYLUCENE_VERSION-src.tar.gz | tar -xz
  48. ENV PREFIX_PYTHON=/usr \
  49. JCC_JDK=/usr/lib/jvm/default-java \
  50. ANT=ant \
  51. JCC='python -m jcc' \
  52. NUM_FILES=10 \
  53. PYTHON=python \
  54. NO_SHARED=1
  55. RUN cd "pylucene-$PYLUCENE_VERSION/lucene-java-$PYLUCENE_VERSION/lucene" && \
  56. ant ivy-bootstrap && \
  57. ant && \
  58. cd ../../../
  59. RUN cd "pylucene-$PYLUCENE_VERSION/jcc" && \
  60. ls -la && \
  61. NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py build && \
  62. NO_SHARED=1 JCC_JDK=/usr/lib/jvm/default-java python setup.py install && \
  63. cd .. && \
  64. make JCC="python -m jcc" ANT=ant PYTHON=python NUM_FILES=8&& \
  65. make install JCC="python -m jcc" ANT=ant PYTHON=python NUM_FILES=8 && \
  66. cd ../../
  67. RUN apt-get remove -y gpg-agent ant jcc build-essential && \
  68. apt-get purge --auto-remove -y && \
  69. apt-get clean
  70. WORKDIR /usr/src
  71. RUN rm -rf pylucene
  72. RUN python -c "import lucene; lucene.initVM()"
  73. # ======================== END OF ADDITIONAL INSTALLATION ========================
  74. WORKDIR /app
  75. COPY . .
  76. </details>

huangapple
  • 本文由 发表于 2020年8月17日 22:30:26
  • 转载请务必保留本文链接：https://go.coder-hub.com/63452974.html
  • java
  • pylucene
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定