2017年1月14日 06:33:17go评论99阅读模式

英文:

mongoimport throwing Random MSpanList_Insert error on mac Sierra 10.12.2

问题

我的目标是将Modulus中的转储导入到我的本地MongoDB中。我有一个脚本，它逐个导入所有的JSON文件。在安装了Sierra之前，这个脚本运行得很好。以下是脚本的内容：

#!/bin/bash
FILES=/Users/username/Documents/ModulusDumps/1-12-17/*
for f in $FILES
do
  filename="${f##*/}"
  basename="${filename%.[^.]*}"
  echo "$basename $filename"
  mongoimport --db meteor --collection $basename --type json --file $f --drop -h 127.0.0.1:3001
done

现在，在安装了Sierra之后，脚本仍然可以运行并且能够处理JSON文件，但是在导入的过程中，我会在随机位置遇到以下错误：

> failed MSpanList_Insert 0x7a0100 0x142f1b79b2cc2 0x0 fatal error:
> MSpanList_Insert
>
> runtime stack: runtime.MSpanList_Insert(0x427610, 0x7a0100)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/mheap.c:692 +0x8f
> runtime.MHeap_Alloc(0x4275a0, 0x1, 0x10000000002, 0xd9e9)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/mheap.c:240 +0x66
> runtime.MCentral_CacheSpan(0x42e118, 0x4ba3e8)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/mcentral.c:85 +0x167
> runtime.MCache_Refill(0x4b5960, 0xc200000002, 0x4ba3e8)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/mcache.c:90 +0xa0
>
> goroutine 54 [running]: runtime.switchtoM()
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:198
> fp=0xc209723878 sp=0xc209723870 runtime.mallocgc(0x5, 0x0, 0x3,
> 0xc20924dff0)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/malloc.go:147 +0x1bb
> fp=0xc209723928 sp=0xc209723878 runtime.rawstring(0x5, 0x0, 0x0, 0x0,
> 0x0, 0x0)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/string.go:195 +0x93
> fp=0xc209723958 sp=0xc209723928
> runtime.slicebytetostring(0xc208275afc, 0x5, 0x49, 0x0, 0x0)
> /usr/local/Cellar/go/1.4.2/libexec/src/runtime/string.go:69 +0x4b
> fp=0xc2097239e8 sp=0xc209723958
> github.com/mongodb/mongo-tools/common/json.maybeUnquote(0xc208275afc,
> 0x5, 0x49, 0x0, 0x0, 0xc208106001)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:1170
> +0x88 fp=0xc209723a28 sp=0xc2097239e8 github.com/mongodb/mongo-tools/common/json.(*decodeState).objectInterface(0xc208625f00,
> 0xe)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:1076
> +0x14e fp=0xc209723ac0 sp=0xc209723a28 github.com/mongodb/mongo-tools/common/json.(*decodeState).valueInterface(0xc208625f00,
> 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:978
> +0x84 fp=0xc209723b10 sp=0xc209723ac0 github.com/mongodb/mongo-tools/common/json.(*decodeState).bsonDInterface(0xc208625f00,
> 0x0, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:1044
> +0x25a fp=0xc209723bf0 sp=0xc209723b10 github.com/mongodb/mongo-tools/common/json.(*decodeState).bsonDocument(0xc208625f00,
> 0x0, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:358
> +0x65 fp=0xc209723c18 sp=0xc209723bf0 github.com/mongodb/mongo-tools/common/json.(*decodeState).unmarshalBsonD(0xc208625f00,
> 0x0, 0x0, 0x0, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:190
> +0x151 fp=0xc209723c70 sp=0xc209723c18 github.com/mongodb/mongo-tools/common/json.UnmarshalBsonD(0xc208275900,
> 0x245, 0x245, 0x0, 0x0, 0x0, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/common/json/decode.go:107
> +0x173 fp=0xc209723ce0 sp=0xc209723c70 github.com/mongodb/mongo-tools/mongoimport.JSONConverter.Convert(0xc208275900,
> 0x245, 0x245, 0x604e, 0x0, 0x0, 0x0, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/json.go:142
> +0x7b fp=0xc209723dd8 sp=0xc209723ce0 github.com/mongodb/mongo-tools/mongoimport.(*JSONConverter).Convert(0xc208ac3b40,
> 0x0, 0x0, 0x0, 0x0, 0x0) <autogenerated>:8 +0xcd fp=0xc209723e28
> sp=0xc209723dd8
> github.com/mongodb/mongo-tools/mongoimport.(*importWorker).processDocuments(0xc2080ac040,
> 0xc2080ac000, 0x0, 0x0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/common.go:380
> +0x166 fp=0xc209723f68 sp=0xc209723e28 github.com/mongodb/mongo-tools/mongoimport.func·002(0xc208086120,
> 0xc2080c6060, 0xc2080d85a0)
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/common.go:268
> +0xa7 fp=0xc209723fc8 sp=0xc209723f68 runtime.goexit() /usr/local/Cellar/go/1.4.2/libexec/src/runtime/asm_amd64.s:2232 +0x1
> fp=0xc209723fd0 sp=0xc209723fc8 created by
> github.com/mongodb/mongo-tools/mongoimport.streamDocuments
> /private/tmp/mongodb20150321-60760-11xdj8i/mongodb-src-r3.0.1/src/github.com/mongodb/mongo-tools/.gopath/src/github.com/mongodb/mongo-tools/mongoimport/common.go:275
> +0x490

还有一些额外的输出...
我整天都在搜索，发现这个错误在Sierra中并不新鲜，而且与Go语言有关。大概是1.4版本不被支持或者不支持Sierra。我已经安装了Go 1.7并进行了测试，但是没有效果。在mongoimport过程中仍然会出现相同的错误。任何帮助将不胜感激！

英文:

My goal is importing a dump from Modulus into my local MongoDB. I have a script that took all of the JSON files and imported them one by one. The script worked great until I installed Sierra. Here is the script:

#!/bin/bash
FILES=/Users/username/Documents/ModulusDumps/1-12-17/*
for f in $FILES
do
  filename=&quot;${f##*/}&quot;
  basename=&quot;${filename%.[^.]*}&quot;
  echo &quot;$basename $filename&quot;
  mongoimport --db meteor --collection $basename --type json --file $f --drop -h 127.0.0.1:3001
done

Now, after the Sierra install, the script still runs through the JSON files and works, but at random places in the import I get the following:

Plus some additional output...
I've been searching all day and found that this error is not new in Sierra and is related to the Go language. Something about 1.4 version not being supported or not supporting Sierra.
I've installed Go 1.7 and tested it, but to no avail. Same error in random places during the mongoimport.
Any help would be appreciated!!

答案1

得分: 1

我在清理数据并在 macOS Sierra 10.12.6 上使用新的 MongoDB 版本 3.7.4 再次运行 'mongoimport' 后解决了类似的问题。

在解决问题之前：
我使用的是 MongoDB 版本 3.0.7（与 macOS Sierra 10.12.6 一起使用）。今天我尝试导入一个大型 JSON 文件时遇到了类似的问题。以下是示例命令和错误：

./mongoimport --host localhost --port 27017 --db mydb --collection access --type json  --file /Users/data.json

2017-08-31T08:08:29.376+0530	connected to: localhost:27017
failed MSpanList_Insert 0x1341998 0x2a98efb0db8fa 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0x433940, 0x1341998)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0x433660, 0x1, 0x10000000009, 0xd9e9)
	/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0x43a9b8, 0xc20c2e9c88)
	/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0x600960, 0x9, 0x8013417f8)
	/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 20 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20c2e9368 sp=0xc20c2e9360
runtime.mallocgc(0x80, 0x0, 0x3, 0xc208531718)
	/usr/local/go/src/runtime/malloc.go:178 +0x849 fp=0xc20c2e9418 sp=0xc20c2e9368
runtime.rawmem(0x80, 0x80)
	/usr/local/go/src/runtime/malloc.go:371 +0x39 fp=0xc20c2e9440 sp=0xc20c2e9418

./mongoimport --version
mongoimport version: 3.0.7
git version: 134c548992e8248c7a7c53777a652cbb2490ab6c

如何解决问题：
我删除了 /data/db/ 下的 MongoDB 数据。根据您的配置，路径可能不同。

sudo rm -rf /data/db/*

然后我从 https://www.mongodb.com/dr/fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-3.4.7.tgz/download 下载并安装了新的 MongoDB 版本 3.7.4。

drwxr-xr-x   8 root  root  272 Aug 31 07:38 mongodb-osx-x86_64-3.0.7
drwxr-xr-x   7 root  root  238 Aug 31 07:39 mongodb-osx-x86_64-3.4.7
lrwxr-xr-x   1 root  root   24 Aug 31 07:39 mongo -&gt; mongodb-osx-x86_64-3.4.7

检查 mongoimport 版本：

./mongoimport  --version
mongoimport version: r3.4.7
git version: cf38c1b8a0a8dca4a11737581beafef4fe120bcd
Go version: go1.7
   os: darwin
   arch: amd64
   compiler: gc
OpenSSL version: OpenSSL 0.9.8zh 14 Jan 2016

我成功地使用 'mongoimport' 3.4.7 导入了大型文件（以 GB 为单位）。

注意：如果由于任何原因无法像我一样清理数据，您可以尝试单独安装新的 MongoDB 并使用新版本的 'mongoimport'。

我还尝试了在清理数据后使用旧版本的 'mongoimport'，但没有成功。使用最新的 'mongoimport' 帮助解决了我的问题。

希望对您有所帮助。

英文:

I solved similar issue after cleaning up the data and running 'mongoimport' again using new mongo version 3.7.4 on macOS Sierra 10.12.6.

Before solving the issue:

I was using mongo version 3.0.7 (with macOS Sierra 10.12.6). I have observed similar issue today when I tried to import a large JSON file. Here is the sample command and error:

./mongoimport --host localhost --port 27017 --db mydb --collection access --type json  --file /Users/data.json

2017-08-31T08:08:29.376+0530	connected to: localhost:27017
failed MSpanList_Insert 0x1341998 0x2a98efb0db8fa 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0x433940, 0x1341998)
	/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0x433660, 0x1, 0x10000000009, 0xd9e9)
	/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0x43a9b8, 0xc20c2e9c88)
	/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0x600960, 0x9, 0x8013417f8)
	/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 20 [running]:
runtime.switchtoM()
	/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20c2e9368 sp=0xc20c2e9360
runtime.mallocgc(0x80, 0x0, 0x3, 0xc208531718)
	/usr/local/go/src/runtime/malloc.go:178 +0x849 fp=0xc20c2e9418 sp=0xc20c2e9368
runtime.rawmem(0x80, 0x80)
	/usr/local/go/src/runtime/malloc.go:371 +0x39 fp=0xc20c2e9440 sp=0xc20c2e9418

./mongoimport --version
mongoimport version: 3.0.7
git version: 134c548992e8248c7a7c53777a652cbb2490ab6c

How it got fixed:
I removed the mongo data under /data/db/*. It will be some other path as per your configuration.

sudo rm -rf /data/db/*

Then I downloaded & installed new mongo version 3.7.4 from https://www.mongodb.com/dr/fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-3.4.7.tgz/download

drwxr-xr-x   8 root  root  272 Aug 31 07:38 mongodb-osx-x86_64-3.0.7
drwxr-xr-x   7 root  root  238 Aug 31 07:39 mongodb-osx-x86_64-3.4.7
lrwxr-xr-x   1 root  root   24 Aug 31 07:39 mongo -&gt; mongodb-osx-x86_64-3.4.7

Check the mongoimport versoin

./mongoimport  --version
mongoimport version: r3.4.7
git version: cf38c1b8a0a8dca4a11737581beafef4fe120bcd
Go version: go1.7
   os: darwin
   arch: amd64
   compiler: gc
OpenSSL version: OpenSSL 0.9.8zh 14 Jan 2016

I was successfully able to import the large file (in GBs) using 'mongoimport' 3.4.7.

Note: if you can't cleanup the data same as I did because of any reason. You can try installing new mongo separately and try 'mongoimport' using new version.

I also tried the 'mongoimport' using older version after cleaning up the data but it didn't work. Using latest 'mongoimport' helped to solve my issue.

Hope it helps.

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

mongoimport在mac Sierra 10.12.2上抛出随机的MSpanList_Insert错误。

问题

答案1

在执行结束时执行操作

what's difference between make and initialize struct in golang?

How can I properly demonstrate concurrency AND parallelism in Go/Golang?

ListenAndServe返回net.DNSError错误：”未提供节点名或服务名”

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论