Currently Being Moderated

Introduction to FPolicy

FPolicy Screening.png

 

 

Worldwide unstructured data in growing at an exponentially rate. IDC predicts that by 2020 amount of digital data can be a mammoth 40 Zettabytes(ZB). Significant percentage of this will be file based unstructured data, which is in fact one of the fastest growing segment of digital data. Unstructured file data is expected to grow at CAGR 61.5% YoY posing significant challenges to Organizations and regulators. Managing this data requires adhering to best practices in the industry and leveraging the expertise in the ecosystem. Clustered Data ONTAP 8.2 enables this through its File Policy framework FPolicy.

 

FPolicy is a file access notification framework which allows file screening of NFS and CIFS accesses.  Introduced in clustered Data ONTAP 8.2 for NetApp scale out architecture this enables a rich set of use-cases working with our selected partners. FPolicy requires that all the nodes, in the cluster, are running Data ONTAP 8.2 or later. FPolicy supports all the SMB versions like SMB1.0 (aka CIFS), SMB 2.0, SMB 2.1 and SMB 3.0 and major NFS versions like NFS V3 and NFS V4.0. 

 

FPolicy natively supports simple file blocking use case which enables administrators to restrict end users to store unwanted files. Example: An admin can block   audio and video files from getting stored in Data Centers and thus saving precious storage resources. This feature blocks files only based on extension and for more advanced features partner solutions have to be considered. The framework generates external on-wire notifications which can either alter the file access path or just report the file access: enabling a diverse set of use-cases. Partner solutions, built around the framework, provides rich set of  use-cases like data governance, File access reporting, User and directory quota, File tiering, File Replication, encryption and many more.

 

The notification framework is platform agnostic and the partner application can run on Windows, UNIX, Linux or Mac platform. The underlying operating system can be either a physical server or a virtual one. FPolicy best practices need to be followed while developing and deploying the solution. FPolicy best practices are part of CIFS best practices Technical Report

 

Kindly be aware that the “Notification Schema” and “Sample Code” provided as part FPolicy SDK are legally protected resources. Currently they are available only to NetApp Technology Partners. Generic FPolicy configuration ONTAPIs are covered in NMSDK 8.2 documentation and available to all NetApp customers and partners.

 

 

 

 

Configuring FPolicy

 

 

The first step to implementing an FPolicy use-case is to configure a resident FPolicy policy on the cluster. This policy can be configured either in cluster scope or Storage Virtual Machine (SVM) scope. When configured in cluster scope they act as template and can be applied to all the SVMs in the cluster. Resident policies are configured either by Command Line Interface (CLI) or by FPolicy APIs, which are documented in NMSDK documentation for clustered Data ONTAP 8.2 clustered mode.

FPolicy Workflow.png

 

 

 

Resident policy on the controller is divided into four containers based on functionality. They are

 

External Engine: This container manages external communication with FPolicy server application.

Events: This container captures information about protocols and file operations monitored for the policy

Policy: Main container which associates different constituents of the policy and provides platform for policy management like policy enabling and disabling

Scope: This container define the storage objects on which the policy will act. Ex: Volumes, sharers, exports and file extensions

 

Workflow for creating resident policy is as shown in the diagram. You need to create external engine or Events before creating a policy. Only when policy is defined you can associate a scope to it.

Once the scope is created policy needs to be enabled with the sequence number. The sequence number helps to define the priority of the policy in a multi-policy environment. With sequence of 1 having highest priority and 10 having the lowest.

 

 

 

 

 

 

 

Native File Blocking

 

 

FPolicy can be used for native file blocking. Native blocking policy can be configure in a few simple steps using command line interface (CLI). Configuring and managing these policies will be supported in future version of System Manager as well. They can be configured either at cluster context or SVM context based on the storage requirement. Let us look at configuring policy in SVM context

 

 

Configuring Policy Engine

External engine defines the IP address and port of the FPolicy server. It defines the characteristics of notification channel like

  • sync or async mode of communication
  • SSL or plain text communication

Native blocking use case doesn't have any dependency on external engine and instead uses a preconfigured template 'native'  in place of external engine

 

 

Configuring Policy Events

we recommend monitoring these file operations for the native file blocking use case.

 

Protocol

File Operations to Monitor

Folder operations to Monitor

CIFS

Create, Open, Rename

NA

NFS V3

Create, Write, Rename, Symlink

NA

NFS V4

Create, Open, Rename, Symlink

NA

 

This command will create a event for monitoring CIFS protocol. You can create containers for NFSV3 and NFSV4 and associate with a single policy

 

clus2::> fpolicy policy event create -vserver <<vserver-name> -event-name <<event-name>> -protocol cifs -file-operations create,open,rename -volume-operation false

 

 

Configuring Policy Container

This is the placeholder that will associate Event and External Engine with Scope.   This has other flags that can be set to define the policy behavior like is-mandatory, allow-privileged-access and others. For Native Blocking use case these fields are irrelevant. In Native blocking use case sequence number is irrelevant and highest priority is assigned to the policy.
This command will create the policy event with preexisting external engine template native.

 

clus2::> fpolicy policy create -vserver <<vserver-name>> -policy-name <<policy-name>> -events <<event-name>> -engine native

 

 

Configuring Policy Scope

Scope will define the storage objects on which the policy will be active. The storage objects can be volumes, CIFS shares or NFS exports. One can choose the file extension that will be monitored by the policy. If you like to monitor all the files you can either live it blank or give '*' . File extension can be a regular expression and support '*' and '?' as special characters. Scope of policy on storage objects can be controlled with the include and exclude lists. Between two lists exclude list has higher priority over include list. 

 

Some operating systems like Mac has folders with extension. Monitoring folders with extension can be controlled with is-file-extension-check-on-directories-enabled option available in advanced mode. Default value of the option is false and operations on folders will be monitored by default. The default response is to block  access; share access will be blocked. To overcome this limitation, while configuring native blocking, we should enforce extension on directories as well by setting the option to true

 

This command will define the policy scope that monitors only file operation on text and MS Doc files. Scope is defined on all shares except share1

Note: Based on the terminal user may not be able to enter special characters in such cases you may have to press escape key before keying in the characters

 

clus2::*> fpolicy policy scope create -vserver <<vserver-name>> -policy-name <<policy-name>> -shares-to-include "*" -shares-to-exclude share1 -file-extensions-to-include "txt,doc*" -is-file-extension-check-on-directories-enabled true

 

Enabling Policy

Policy can be enabled with a simple command using the available sequence number. Sequence number is irrelevant for native blocking use case but an arbitrary value from 1-10 has to be provided.  Once enabled policy should be configured and running.

 

clus2::> fpolicy enable -policy-name <<policy-name>> -sequence-number <<sequence number>>

 

 

 

Enhancements to 7-Mode

 

Significant enhancements are made to FPolicy implementation in clustered Data ONTAP compared to earlier implementation in Data ONTAP 7-Mode. They can be broadly classified on these lines

  • No-dependency on resources like pblks
  • File notification are sent in XML over TCP format, making it both platform independent and less taxing on network
  • Ability to clearly set policy priority in multiuse-case scenario
  • Ability to monitor directory extension. Ex: Mac environment
  • Ability to apply back pressures to reduce load on Servers
  • Ability to filter the notification on SVM side that enables optimal utilization of resources and better overall solution throughput.
  • Securing the notification channel through SSL. Notification channel can be plain text, dual side or single side encryption
  • Enhanced FPolicy safeguards to manage client outages in case of Network or server disruption

 

 

Partner Solutions

 

For some of the advanced use-cases we integrate with our partners to provide an end to end solution and incorporate our best practices. Partner solution leverage FPolicy framework to offer additional use cases on top of the native use cases provided by Data ONTAP. The solutions can be in the space of security, storage management, access reporting, archiving, File Replication and others. Some of our active partners in this space

  • Varonis Data Governance and Data Analytics solution DatAdvantage
  • NTP Software QFS for Quota and File Screening
  • Many more to come…

 

 

Reference

 

  • FPolicy best practices will be made available as part of CIFS best practices
  • Refer FPolicy section in File Access And Protocol Management Guide for detail description of FPolicy CLI commands
  • For information on FPolicy APIs kindly refer to NMSDK document for clustered Data ONTAP 8.2

Comments

Filter Blog

By author:
By date:
By tag: