NetApp/harvest v24.11.0 on GitHub

24.11.0 / 2024-11-06 Release

📌 Highlights of this major release include:

💎 New dashboards:
- SnapMirror Destinations Dashboard which displays relationship details from the destination perspective.
- Vscan Dashboard which shows SVM-level and connection scanner details.
⭐ Several of the existing dashboards include new panels in this release:
- SnapMirror dashboard now includes relationship details from the source perspective and has been renamed to "ONTAP: SnapMirror Sources".
- Health Dashboard's emergency events panel now includes all emergency EMS events from the last 24 hours.
- Network Dashboard
  - Includes Link Aggregation Group (LAG) metrics
  - Adds Ethernet port details
- s3 Object Storage dashboard includes panels for s3 metrics for SVM.
- Tenant Dashboard
  - Adds Tenant/Bucket Capacity Growth Chart
  - Includes average size per object details for each bucket
- Metadata Dashboard includes a panel displaying the number of instances collected.
- Power Dashboard includes a new "Average Power Consumption (kWh) Over Last Hour" panel.
- SVM Dashboard now features panels for logical space and physical space at the SVM level.
- Volume Deep Dive dashboard includes "Other IOPs" panel.
🚀 Performance Improvements:
- Reduced memory footprint by optimizing memory allocations when serving metrics.
- Reduced API calls when using the RestPerf collector.
Harvest supports Top clients metrics collection. More details.
Harvest supports recording and replaying HTTP requests.
Harvest now provides a FIPS-compliant container image, available as a separate image (ghcr.io/netapp/harvest:24.08.0-1-fips).
Grafana import allows rewriting the cluster label during import.

Announcements

‼️ IMPORTANT NetApp moved their communities from Slack to Discord, please join us there!

‼️ IMPORTANT If using Docker Compose and you want to keep your historical Prometheus data, please
read how to migrate your Prometheus volume

💡 IMPORTANT After upgrade, don't forget to re-import your dashboards, so you get all the new enhancements and fixes. You can import them via the 'bin/harvest grafana import' CLI, from the Grafana UI, or from the 'Maintenance > Reset Harvest Dashboards' button in NAbox.

Thanks to all the awesome contributors

🤘 Thanks to all the people who've opened issues, asked questions on Discord, and contributed code or dashboards
this release:

🌱 This release includes 36 features, 24 bug fixes, 7 documentation, 7 performance, 1 testing, 3 styling, 5 refactoring, 9 miscellaneous, and 15 ci pull requests.

🚀 Features

Tenant Dashboard Buckets Panel Should Include (#3101)
Use Docker Buildx Secret For Token (#3108)
Enable Pprof Endpoints On Localhost (#3110)
Generate Fips Compliant Container Image For Harvest (#3113)
Support Ifgroup Level Throughput Metrics (#3117)
Harvest Should Include A Vscan Dashboard (#3121)
Vscan Dashboard Should Include Topk (#3127)
Top Clients Metrics Collection (#3132)
Adding Panels For Ontaps3svm Object (#3134)
Grafana Import Should Allow Rewriting Cluster Label (#3135)
Replace Zerolog With Slog (#3146)
Harvest Should Include Time-Series Panels For Tenants And Buckets (#3147)
Send The Harvest Version To Ontap (#3152)
Replace Zerolog With Slog (#3164)
Add Documentation For Plugin-Generated Metrics And Enable Ci (#3169)
Add Instances Collected Panel To Metadata Dashboard (#3178)
Harvest Should Use Slogs Text Format By Default (#3179)
Add "Average Power Consumption (Kwh) Over Last Hour" Panel To Power Dashboard (#3180)
Replacing connector webhook with MS workflow (#3183)
Handle Url Limit In Rest (#3186)
Keyperf Collector Templates (#3194)
Harvest Rest And Restperf Collectors Should Support Batching (#3195)
Add Top Svm By Space In Svm Dashboard (#3200)
All Harvest Dashboards Should Include Tags (#3202)
Support Destination/Source Level View - Parity With Sm (#3204)
Add Other Ops Panel In Volume Deep Dive Dashboard (#3209)
Add Nfs Templates For Keyperf Collector (#3215)
Adding Snapmirror Sources dashboard - 1 (#3216)
Keyperf Collector Templates (#3219)
Adding Ethernet Port Table From Netport Template (#3221)
Fail Ci When There Are Errors In Prometheus Or Grafana (#3232)
Log Cluster Name And Version With Poller Metadata (#3234)
Harvest Should Support Recording And Replaying Http Requests (#3235)
Add Emergency Events To Health Dashboard (#3238)
Add Keyperf Metric Docs (#3240)
Improve Harvest Memory Logging (#3244)
Doctor should handle embedded exporters (#3258)

🐛 Bug Fixes

Handled Non Exported Qtrees In Template (#3105)
Handled Nameservices In Svm Zapi Plugin (#3124)
Fix Disk Count In Disk Dashboard (#3126)
Handled Quota Index Key In Rest Template With Tests (#3128)
Vscan Panels Throws 422 Error (#3133)
Correcting The Alert Rule Expression For Required Labels (#3143)
Svm Dashboard - Volume Capacity Row Ordering (#3158)
Fsa History Data Should Work When Multi Select (#3159)
Do Not Log Stdout When A Credential Script Fails (#3163)
Remove '*' As 'All' Option In Workload Dropdown On Workload Dashboard (#3165)
Bin/Harvest Rest Should Read Credentials Before Fetching Data (#3166)
Remove Embedded Shelf Power From Total Power In Series Panel To Match Stats Panel (#3167)
Volume_aggr_labels Should Not Include Uuid Label (#3171)
Add Embedded Shelf Type For Power Calculation (#3174)
Using Instancename Instead Of Volname In Fabricpool Perf (#3175)
Correct Failed State In Workflow (#3190)
Handled Flexgroup Based On Volume Config Call (#3199)
Filter By Svm, Volume In Sm Destination Dashboard (#3220)
Remove _Labels From Metric Docs (#3222)
Update Datacenter And Cluster Variables In Dashboards (#3227)
Don't Double Export Aggregate Efficiency Metrics (#3230)
Update Keyperf Collector Static Counter File Path (#3241)
Fix Numbering In Quickstart (#3249)
Fix Value Mapping In Tenant Dashboard (#3253)
Rename volume latency in keyperf (#3261)

📕 Documentation

Fix Typo In Docs (#3112)
Clarify Ipv6 Support (#3119)
Topclients Plugin Document (#3151)
Add More Credential Script Troubleshooting Steps (#3154)
Remove Qos Service Latency Counter From Metric Docs (#3188)
Add Space To Datacenter Dashboard Title (#3225)
Update Release Months To Match Harvest Release Cadence (#3236)
Update KeyPerf metric docs (#3260)

⚡ Performance

Reduce Allocs When Reading Credential Files (#3111)
Reduce Allocs In Prometheus Render (#3168)
Reduce Allocations When Serving Prometheus Metrics (#3172)
Reduce Poller Footprint By Not Collecting Smb Histogram Metrics … (#3177)
Restperf Collectors Should Only Run Pollinstance For Workloads (#3207)
Reduce Allocs When Serving Metrics (#3208)
Reduce Allocs When Rendering Metrics (#3214)

🔧 Testing

Add Authtoken With Password Testcase (#3176)

Styling

Ensure Slogging Uses Attributes Only (#3197)
Add Debug Logs For Volume Plugin (#3233)
Bring Harvest.cue Up To Date (#3256)

Refactoring

Rename Volumetopclients Maxvolumecount To Max_volumes (#3141)
Improve Logging (#3182)
Change Auth Credential Script Logging To Debug (#3191)
Improve Slog Error Logging (#3198)
Truncate Href When Logging (#3245)

Miscellaneous

Update All Dependencies (#3096)
Merge Release/24.08.0 Into Main (#3099)
Remove Zerolog Stack Calls (#3118)
Update Module Github.com/Shirou/Gopsutil/V4 To V4.24.8 (#3129)
Update All Dependencies (#3139)
Remove Calls To Msgf("") (#3144)
Update All Dependencies (#3196)
Update Prometheus Version V2.55.0 (#3223) (#3226)
Update Module Github.com/Shirou/Gopsutil/V4 To V4.24.10 (#3250)

🔨 CI

Bump Go (#3106)
Fix Container Image Push Order (#3116)
Bump Go (#3138)
Add Metric Generate Docs To Ci (#3150)
Fix Ci Datacenter Name (#3170)
Replacing Connector Webhook With Ms Workflow (#3183)
Bump Gopsutil (#3185)
Bump Go (#3187)
Add Missing Dependency Of Purego (#3189)
Add Trivy To Ci (#3217)
Embusalacchi Has Signed The Ccla (#3224)
Add Docker Login To Ci (#3237)
Harvest Should Lint Metrics With Promtool (#3246)
Use Zizmor To Find Security Issues In Github Actions Setups (#3247)
Keyperf Collector Does Not Exist In Harvest Version 22.11 (#3248)

NetApp/harvest v24.11.0 24.11.0 on GitHub