Skip和Limit用于Mongo聚合分页。

huangapple go评论76阅读模式
英文:

Skip and Limit for pagination for a Mongo aggregate

问题

我正在使用Flask(Python框架)和flask-paginate(仅供参考)来处理分页。

我能够对以下的“find”查询实现分页:

from flask_paginate import Pagination
from flask_paginate import get_page_args

def starting_with_letter(letter):
    page, per_page, offset = get_page_args()
    collection_name = letter.lower() + '_collection'
    words = db[collection_name]
    data_db = words.find()
    data = data_db.limit(per_page).skip(offset)  # 在这里我已经实现了限制和跳过
    pagination = Pagination(page=page, total=data.count(), per_page=per_page, offset=offset, record_name='words')
    return render_template('startingwords.html', data=data, pagination=pagination)

但是对于聚合操作,我无法实现相同的分页:

def test():
    page, per_page, offset = get_page_args()
    cursor_list = []  # 在循环迭代中附加每个游标
    collections = db.collection_names()
    for collection in collections:
        cursor_objects = db[collection].aggregate([
            {
                "$match": {
                    "$expr": {"$eq": [{"$strLenCP": "$word"}, 6]}
                }
            },
            {"$skip": offset},
            {"$limit": per_page}
        ])
        for cursor in cursor_objects:
            cursor_list.append(cursor)
    pagination = Pagination(page=page, total=len(cursor_list), per_page=per_page, offset=offset, record_name='words')
    return render_template('lettersearch.html', data=cursor_list, pagination=pagination)

结果显示为:

Skip和Limit用于Mongo聚合分页。

在单个页面上显示了所有的39个结果

点击“第2页”后显示:

Skip和Limit用于Mongo聚合分页。

注意:默认情况下,flask-paginate将“per_page”初始设置为10,“offset”初始设置为0。

在查阅了许多链接之后,我尝试过:

将“skip”和“limit”放在“match”之前,这是错误的

还了解到“limit”总是在“skip”之后。

我被卡住了,请帮忙。

英文:

I am working on pagination in flask(Python framework) using flask-paginate (just for ref)

I am able to achieve pagination for just a find query as below:

from flask_paginate import Pagination
from flask_paginate import get_page_args

def starting_with_letter(letter):
    page, per_page, offset = get_page_args()
    collection_name=letter.lower()+'_collection'
    words=db[collection_name]
    data_db=words.find()
    data=data_db.limit(per_page).skip(offset) '''Here I have achieved the limit and skip'''
    pagination = Pagination(page=page, total=data.count(),per_page=per_page,offset=offset,record_name='words')
    return render_template('startingwords.html',data=data,pagination=pagination)

But I am not able to do the same for the aggregate here:

def test():
    page, per_page, offset = get_page_args()
    cursor_list=[]  '''appending each cursor in iteration of for loop '''
    collections=db.collection_names()
    for collection in collections:
        cursor_objects = db[collection].aggregate([
                {
                    "$match": {
                        "$expr": {"$eq": [{"$strLenCP": "$word"}, 6]}
                    }
                },
                            {"$skip": offset},    
                            {"$limit": per_page}

            ])
        for cursor in cursor_objects:
            cursor_list.append(cursor)
    pagination = Pagination(page=page, total=len(cursor_list),per_page=per_page,offset=offset,record_name='words')
    return render_template('lettersearch.html',data=cursor_list,pagination=pagination)

The results are displayed as :

Skip和Limit用于Mongo聚合分页。

Here all the 39 results are shown at single page

On hitting page 2 it showed :

Skip和Limit用于Mongo聚合分页。

Note: By default flask-paginate sets initially per_page as 10 and offset as 0

after referring many links i have tried:

placing skip and limit above match which is wrong any way

Also learnt that limit is always followed by skip

I am stuck with this, Any help is appreciated

答案1

得分: 1

你的问题不在于 skip()limit(),它们运行正常。问题出在你的整体逻辑上,你在第一个循环中迭代了所有 39 个集合,然后将聚合的每个结果附加到 cursor_list

我无法理解你试图做什么的逻辑,因为第一个示例在查找单词集合中的内容,而第二个示例在所有集合中查找单词字段。但话虽如此,你可以简化你的方法,如下所示:

offset = 0
per_page = 10
collections = db.list_collection_names()
#
# 在集合数组上添加一些逻辑以筛选所需内容
#
print(collections[offset:offset+per_page])

根据评论进行编辑,以下是执行此操作的完整工作示例函数。不需要聚合查询,这会增加复杂性。

from pymongo import MongoClient
from random import randint

db = MongoClient()['testdatabase1']

# 设置一些数据
for i in range(39):
    coll_name = f'collection{i}'
    db[coll_name].delete_many({})  # 小心使用;仅用于测试;这将删除你的数据
    for k in range(randint(0, 2)):
        db[coll_name].insert_one({'word': '123456'})

# 主要函数
def test(offset, per_page, word_to_find):
    found = []
    collections = db.list_collection_names()
    for collection in sorted(collections):
        if db[collection].find_one({word_to_find: { '$exists': True}}) is not None:
            found.append(collection)

    print(found[offset:offset+per_page])

test(offset=0, per_page=10, word_to_find='word')

希望这能帮助你理解代码并解决问题。

英文:

Your issue is not with the skip() and limit(); that is working fine. The issue is with your overall logic; you are iterating all 39 collections in the first loop and then appending each result of the aggregation to cursor_list.

I can't figure out the logic of what you are trying to do, as the first example is looking in a words collection and second is looking in all collections for a word field; with that said, you can likely simplify your approach to something like:

offset = 0
per_page = 10
collections = db.list_collection_names()
#
# Add some logic on the collections array to filter what is needed 
#
print(collections[offset:offset+per_page])

EDIT to reflect comments. Full worked example of a function to perform this. No need for an aggregation query - this adds complexity.

from pymongo import MongoClient
from random import randint

db = MongoClient()['testdatabase1']

# Set up some data
for i in range(39):
    coll_name = f'collection{i}'
    db[coll_name].delete_many({}) # Be careful; testing only; this deletes your data
    for k in range (randint(0, 2)):
        db[coll_name].insert_one({'word': '123456'})

# Main function
def test(offset, per_page, word_to_find):
    found = []
    collections = db.list_collection_names()
    for collection in sorted(collections):
        if db[collection].find_one({word_to_find: { '$exists': True}}) is not None:
            found.append(collection)

    print(found[offset:offset+per_page])

test(offset=0, per_page=10, word_to_find='word')

huangapple
  • 本文由 发表于 2020年1月3日 13:14:44
  • 转载请务必保留本文链接:https://go.coder-hub.com/59573513.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定