#mongodb #aggregation-framework
#mongodb #агрегация-фреймворк
Вопрос:
Я пытаюсь выяснить, как удалить дубликаты на основе URL-адреса, поскольку агрегированный запрос может дважды сопоставлять один и тот же документ, если, скажем, «APPL» и «TSLA» находятся в stocks одном документе и включены в него.
const res = await ReadSchema.find({ 'tickers.symbol': { $in: stocks } })
.select(['-content', '-htm'])
.sort({ createdAt: -1 })
.limit(100)
.filter((v, i, a) => a.findIndex((t) => t.url === v.url) === i);
Я делаю это js после результатов, но я бы предпочел сделать это до .limit(100) , чтобы удалить повторяющиеся URL-адреса, прежде чем я получу первые 100.
Вот пример документа:
{
"_id" : ObjectId("6001eb414110e6001cb5b226"),
"summary" : {
"topics" : [
"buds",
"pro",
"galaxy",
"anc",
"live",
"sound",
"samsung",
"noise",
"ear",
"app"
],
"sentiment" : 0,
"words" : 0,
"difficulty" : 0,
"minutes" : 0
},
"views" : 1,
"title" : "Samsung takes on Apple's AirPods Pro with new Galaxy Buds Pro",
"byline" : "Alex Perry",
"dir" : null,
"content" : "<div id="readability-page-1" class="page"><section>nn<p>Sometimes making things less fun is for the best. Case in point: The Samsung Galaxy Buds Pro.amp;nbsp;</p>n<p>Just a handful of months after Samsung graced us with the delightfully bean-shaped (and genuinely good) <a href="https://mashable.com/review/samsung-galaxy-buds-live-review/">Galaxy Buds Live</a>, we've already got a more, uh, "professional" follow-up. At $199.99, the Galaxy Buds Pro are only marginally more expensive than the Buds Live, but offer a totally different look and a few extra features, largely revolving around active noise cancellation.</p>n<p>The end result is something that should please Samsung fans (and Android users in general). But is there any reason to spend that extra $30 instead of getting the <em>already totally fine</em> Galaxy Buds Live?amp;nbsp;</p>n<div>nn<div>n<div>n<div>n<p>nMuch more comfortable than Buds Live • Improved ANC • Sound quality is still great • Strong app-based customizationn</p></div>n<div>n<p>nVoice detection feature is a dud • ANC struggles with outdoor bustlen</p></div>nn</div>nnnn</div>n</div>nn<h2>The Good: Improved ANC and comfort, SmartThings Find, sound quality</h2>n<p>The most noticeable difference between the Buds Live and the Buds Pro from the jump is their appearance. As we've discussed, the Buds Live were shaped like something you'd find in a burrito and, on a practical level, let a decent amount of outside noise in due to the lack of a seal of any kind. Their ANC features did a little bit to offset this, but it wasn't good enough. It was pretty sweet that they attracted next to no earwax because they didn't actually go inside the ear, though.</p>n<p>The Buds Pro aren't nearly as eye-catching but they make up for it in the ways that actually matter. Like <a href="https://mashable.com/review/airpods-pro-review/">Apple's AirPods Pro</a> equivalent, the Buds Pro have rubber ear cups that you insert into the ear canal with a rounded touch panel that sits outside featuring basic playback controls. Tap once to pause, tap twice to skip ahead, and so on. Samsung is shipping the Buds Pro with three different ear cup sizes to accommodate different ear shapes, as well.</p>n<div>n<div>nn<figure><div>n <p><img alt="Galaxy Buds Live" data-fragment="m!0fdc" data-image="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__1200x2000.png?signature=NZalyPHmMKjSvE_-Md19lx1Ovfk=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-micro="1" loading="lazy" naturalwidth="950" sizes="(max-width: 300px) 280w,(max-width: 400px) 356w,(max-width: 700px) 638w,950w" src="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__950x534.png?signature=0P5F2WHmrx7IopVdxva_M-CrBCs=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" src_type="png" srcset="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__280x157.png?signature=KLY0Uk8B1s5bL03gF9ehpT1HFM4=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 280w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__356x205.png?signature=Z-q_8wMfWcP-cbOWGMcJOcIcps0=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 356w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__638x368.png?signature=dUn8anfa1mPCeLNJAwiejq4X83E=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 638w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591715%2F873bf2ac-34d3-4a4a-b2cd-f3befe3f05fe.png%2Ffull-fit-in__950x534.png?signature=0P5F2WHmrx7IopVdxva_M-CrBCs=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 950w">n</p></div>nnn<figcaption><p>Galaxy Buds Liven</p></figcaption><p>Image: alex perry / mashablen</p></figure>n</div>n<div>nn<figure><div>n <p><img alt="Galaxy Buds Pro" data-fragment="m!a70e" data-image="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__1200x2000.jpg?signature=dFshI8fkvoBN6_Rg8ZYPqfRwxb4=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-micro="1" loading="lazy" naturalwidth="950" sizes="(max-width: 300px) 280w,(max-width: 400px) 356w,(max-width: 700px) 638w,950w" src="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__950x534.jpg?signature=SpjFrkhxftv8Z4eGTzVzosVqvhI=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" src_type="jpg" srcset="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__280x157.jpg?signature=HXfNokeRqH-JA92c1-GP_w-fQlI=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 280w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__356x205.jpg?signature=d2iT0s-p_vsJpWa0pK1kHpbm6Qg=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 356w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__638x368.jpg?signature=5eWyZZdHOKWwE4y2-2ogtFJHUg4=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 638w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591722%2Fb72cbc58-58fc-484c-8a54-fae6c2306ebc.jpg%2Ffull-fit-in__950x534.jpg?signature=SpjFrkhxftv8Z4eGTzVzosVqvhI=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 950w">n</p></div>nnn<figcaption><p>Galaxy Buds Pron</p></figcaption><p>Image: alex perry / mashablen</p></figure>n</div>n</div>n<p>From a purely physical standpoint, the Buds Pro are fundamentally a step up from the Buds Live because of how much better they feel in the ear. The "beans" fit snugly and were never a threat to fall out, but the insides of my ears didn't appreciate long sessions with them, and signaled that with minor aching. By virtue of keeping hard plastic away from the sensitive skin inside my ears, the Buds Pro were <em>significantly</em> more comfortable for lengthy periods of time.amp;nbsp;</p>n<p>The ear tips not only make the Buds Pro more comfortable, but they also do a much better job of blocking unwanted outside noise than the Buds Live's open form factor did. While the ANC here still isn't perfect (more on that in a bit), it's a real improvement. When it's enabled and you're not in close proximity to chaos, you'll be able to listen to your music or podcasts in peace.</p>n<p>Just like the Buds Live, Samsung's newest earbuds can connect to the <a href="https://play.google.com/store/apps/details?id=com.samsung.android.app.watchmanageramp;amp;hl=en_USamp;amp;gl=US" target="_blank">Galaxy Wearable</a> Android app for all of your settings and customization needs. That app is Android-only, so iPhone owners should probably buy some AirPods instead. Within the Wearable app you can set ANC to high or low (I didn't notice a huge difference between the two), turn it off entirely, or turn on an ambient sound mode that uses the on-board microphones to amplify sounds around you in situations where you might need to hear what people are saying.amp;nbsp;</p>n<p>Beyond that, you can adjust what the long-press command on the earbuds' outer panels does (taps are set in stone), choose from a handful of different equalizer options like "bass boost" and "treble boost," or even shift the audio balance to favor the left or right side if your hearing is better in one ear than the other. This amount of customization was a strong aspect of the Buds Live and it's still strong here, as well. It isn't super different, but it's appreciated nonetheless.</p>n<p>Another side of the Buds Pro that picks up where Buds Live left off is sound quality. I was really impressed with how deep and rich the sound was in the "beans" despite their unusual shape. That hasn't changed with the shift to a more traditional build. The audio output here isn't dramatically better than what I experienced in the "beans," but the 11mm woofer and 6.5mm tweeter inside do an excellent job of making songs sound like they should.amp;nbsp;</p>n<figure><div>n <p><img alt="The Galaxy Wearable app" data-fragment="m!beae" data-image="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Foriginal.jpg?signature=mVprQdLxhltKq-tGPj21dYhKOzk=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-micro="1" loading="lazy" naturalwidth="950" sizes="(max-width: 300px) 280w,(max-width: 400px) 356w,(max-width: 700px) 638w,950w" src="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Ffull-fit-in__950x534.jpg?signature=cZGICWc0BfWlzHT4dBKEa7BDPos=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" src_type="jpg" srcset="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Ffull-fit-in__280x157.jpg?signature=I9LxxqU9xNEBTe6ZDZ3x8Pf6MFo=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 280w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Ffull-fit-in__356x205.jpg?signature=Vxxu-64ZaXie39nPyghsfKgryA0=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 356w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Ffull-fit-in__638x368.jpg?signature=6fP9FBkiCqJdSAbT6qOBW6EOT2s=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 638w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591421%2Fc7dad555-1148-4700-8268-77002a63f642.jpg%2Ffull-fit-in__950x534.jpg?signature=cZGICWc0BfWlzHT4dBKEa7BDPos=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 950w">n</p></div>nnn<figcaption><p>The Galaxy Wearable appn</p></figcaption><p>Image: alex perry / mashablen</p></figure><p>Instruments and vocals at the front of the mix are appropriately prominent, while aspects of a song that fill out the sound without calling much attention to themselves are fully audible if you focus on them. Some wireless earbuds can flatten songs out due to small, chintzy speakers, but the Buds Pro avoid that particular pitfall. I would still say the <a href="https://mashable.com/review/bose-quiet-comfort-earbuds-review/">Bose QC Earbuds</a> are <em>slightly</em> richer in sound, but they're also $80 more expensive.amp;nbsp;</p>n<p>Battery life is also acceptable, if not remarkable. People who like to use ANC will notice battery drain, as the Buds Pro are rated for about five hours of ANC listening time with an additional 13 hours in the charging case, which is in line with my experience. Turn ANC off and you should get closer to eight hours on a single charge, so if you can live without that key feature, you'll have a better time.</p>n<p>Oh, and one last thing: Galaxy Buds Pro work with Samsung's recently launched device tracking feature within the <a href="https://play.google.com/store/apps/details?id=com.samsung.android.oneconnectamp;amp;hl=en_USamp;amp;gl=US" target="_blank">SmartThings app</a>. SmartThings Find will show you a GPS location for your Buds Pro even if they're off. If they get lost or stolen, this is huge. Simply open the SmartThings app while the Buds Pro are paired to the phone to pair them to the app. It's quick, easy, and eminently useful.</p>n<h2>The Bad: Sensitive voice detection, ANC is better indoors than outdoors</h2>n<figure><div>n <p><img alt="The case is almost identical to the Buds Live case." data-fragment="m!6d74" data-image="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Foriginal.jpg?signature=c1GLO0Mcnadapz4KHB3k_jiZgB0=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-micro="1" loading="lazy" naturalwidth="950" sizes="(max-width: 300px) 280w,(max-width: 400px) 356w,(max-width: 700px) 638w,950w" src="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Ffull-fit-in__950x534.jpg?signature=qMk3EQVFbzAZSCBsJrTc1Hpg5V0=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" src_type="jpg" srcset="https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Ffull-fit-in__280x157.jpg?signature=sKmiyBUFQdAvwwN_hQDYwFSCOGo=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 280w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Ffull-fit-in__356x205.jpg?signature=MrGzmcAVT-ha4f0pyB_cjLjtaV8=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 356w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Ffull-fit-in__638x368.jpg?signature=K03SKbqjuUqgjS0hTi-xENqm3Js=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 638w,https://mondrian.mashable.com/uploads%2Fcard%2Fimage%2F1591414%2F2bf3f78c-75f9-4720-af0b-0e8494231004.jpg%2Ffull-fit-in__950x534.jpg?signature=qMk3EQVFbzAZSCBsJrTc1Hpg5V0=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com 950w">n</p></div>nnn<figcaption><p>The case is almost identical to the Buds Live case.n</p></figcaption><p>Image: alex perry / mashablen</p></figure><p>Unfortunately, one of the more distinct ANC-centric features of the Buds Pro was a dud for me. Samsung is touting the "voice detect" feature (enabled in the Galaxy Wearable app) as a way to get through your day without constantly turning ANC on or off, but I ended up keeping the feature disabled entirely.amp;nbsp;</p>n<p>The way it works is that the Buds Pro can detect when you start speaking, at which point they will turn whatever you're listening to down and turn on the ambient noise mode so you can theoretically carry a conversation for a moment. Once you stop speaking for 10 seconds, everything goes back to the way it was. My issue is that it reacted a little too sensitively to things like sighing and clearing my throat, which would then ruin a song I was listening to for 10 seconds.amp;nbsp;</p>n<p>I wish there were a way to shorten the time before ambient sound turns off, but for now, 10 seconds is the only option you have. I can see people getting use out of this, but personally, I found it more useful to long press one of the earbuds to manually turn ambient sound mode on or off.</p>n<p>The only other minor criticism I have is less of a specific problem with the Buds Pro and more of a general issue with noise-cancelling wireless earbuds. Samsung's ANC here is better than it was in the previous model, but it can still let too much noise in when you're walking down a busy street. In other words, you can drown out the noise of an office or an apartment, but cars whizzing by you will still make noise. Use <a href="https://mashable.com/review/bose-noise-cancelling-700-headphones-review/">over-ear noise-cancelling headphones</a> if you want to <em>really</em> escape from the world.</p>n<h2>The Verdict</h2>n<p>If I owned a Samsung smartphone, the Galaxy Buds Pro would be a no-brainer for me. They work just as seamlessly with something like a Galaxy Note 20 (which I used for testing) as AirPods do with an iPhone, provide terrific sound quality and noise cancellation, and can sit in my ears for hours at a time without bothering me. The improved ANC and additional comfort are enough to make me wholeheartedly recommend these over the Galaxy Buds Live for Android owners.</p>n<p>That said, I wish the ANC did a better job of muting louder outdoor noises, but maybe that's a few years away. It's also unfortunate that the voice detection feature can be more of a hindrance than a help, at least in my experience. Despite those minor issues, as well as middling battery life when ANC is turned on, the Galaxy Buds Pro are a worthy successor to the Galaxy Buds Live and something Android owners should consider.</p>n<p>With all due respect to the "beans," this is a case where boring is better.</p>n<h2>WATCH: Unboxing the BTS Edition Samsung Galaxy Buds </h2>n<div data-source="blueprint" data-template="embedded" data-video="O4oKA2YZnd">n<div>n <p><img alt="Uploads%2fvideo uploaders%2fdistribution thumb%2fimage%2f95136%2f96587d2f 7dea 48ef 87c8 0ed74bd00239.png%2f930x520.png?signature=dkg2rtrnetqd20fsap ou2byynm=amp;amp;source=https://blueprint api production.s3.amazonaws" src="https://mondrian.mashable.com/uploads%2Fvideo_uploaders%2Fdistribution_thumb%2Fimage%2F95136%2F96587d2f-7dea-48ef-87c8-0ed74bd00239.png%2F930x520.png?signature=dkg2rtRNEtQd20FSap-Ou2bYYnM=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-fragment="m!3c0b" data-image="https://mondrian.mashable.com/uploads%2Fvideo_uploaders%2Fdistribution_thumb%2Fimage%2F95136%2F96587d2f-7dea-48ef-87c8-0ed74bd00239.png%2Foriginal.png?signature=b1YV-6FyrAnr5O_ELQa8Vr7FPPE=amp;amp;source=https://blueprint-api-production.s3.amazonaws.com" data-micro="1">n</p></div>nn</div>nnnn</section></div>",
"length" : 8417,
"excerpt" : "Samsung's newest earbuds are less fun to look at, but better to use.",
"siteName" : "Mashable",
"tickers" : [
{
"_id" : ObjectId("6001eb414110e6001cb5b227"),
"name" : "Apple Inc.",
"symbol" : "AAPL",
"exchange" : "nasdaq"
}
],
"cryptos" : [
{
"_id" : ObjectId("6001eb414110e6001cb5b228"),
"name" : "Strong",
"symbol" : "STRONG"
},
{
"_id" : ObjectId("6001eb414110e6001cb5b229"),
"name" : "Tap",
"symbol" : "XTP"
}
],
"meta" : {
"title" : "Samsung takes on Apple's AirPods Pro with new Galaxy Buds Pro",
"thumb" : "https://mondrian.mashable.com/2021%2F01%2F15%2Fdd%2Fac21db42899b4219ab9b4e35fc74c043.002a7.jpg%2F1200x630.jpg?signature=I4q1zJF08njPyxmiZgxnHrSsTsM="
},
"url" : "https://mashable.com/review/samsung-galaxy-buds-pro-review/",
"host" : "mashable.com",
"createdAt" : ISODate("2021-01-15T19:21:37.931Z"),
"updatedAt" : ISODate("2021-01-15T19:21:37.931Z"),
"__v" : 0
}
Комментарии:
1. Пожалуйста, предоставьте несколько примеров документов
2. Добавлено. Мне нужно отсеять дубликаты
.url
Ответ №1:
Вы можете использовать $group этап за $match этапом,
$groupиспользуяurlи получите первый корневой документ$$ROOT, это вернет document вrootполе
{
$group: {
_id: "$url",
root: { $first: "$ROOT" }
}
}
- это необязательно, если вы хотите заменить этот
rootдокумент на root, попробуйте$replaceRoot,
{ $replaceRoot: { newRoot: "$root" } }
Ваш окончательный запрос будет,
const res = await ReadSchema.aggregate([
{ $match: { "tickers.symbol": { $in: stocks } } },
{
$group: {
_id: "$url",
root: { $first: "$ROOT" }
}
},
{ $sort: { "root.createdAt": -1 } },
{ $limit: 100 },
{
$project: {
content: 0,
htm: 0
}
},
{ $replaceRoot: { newRoot: "$root" } }
])
Комментарии:
1. время ожидания запроса истекло.
2. я перемещаю fd $ project вверх и добавляю
.allowDiskUse(true)в конец запроса.3. хорошо, хорошим может быть
contentполе с большим количеством содержимого.
Ответ №2:
await ReadSchema.aggregate([
{ $match: { 'tickers.symbol': { $in: stocks } } },
{
$project: {
content: 0,
htm: 0,
},
},
{
$group: {
_id: '$url',
root: { $first: '$ROOT' },
},
},
{ $sort: { 'root.createdAt': -1 } },
{ $limit: 50 },
{ $replaceRoot: { newRoot: '$root' } },
]).allowDiskUse(true);
```