Solving the Challenges of Record Security with Record Center

As we look back at the typical challenges of Records Management, be it the ability to implement a non-invasive content ingress and approval processes, ensuring that the entire record lifecycle is properly managed—inclusive of disposition, or ensuring that users are able to find the content they need to do their jobs in as few clicks as possible, there’s one remaining pain point that has the potential to bring your entire records management strategy to its knees… security. The structure of your records management solution drives security, but security also often drives structure, so what comes first? Thankfully, Record Center solves these problems for you and stores your records in a structure designed to support your specific security needs.

Storage vs. Security Models

When first configuring a new Record Center instance, you’re asked to select both a Storage Model and a Security Model. These two options work in concert with one another to tell Record Center at what granularity you want to secure your records, and to ensure that they are stored using a structure that supports and enables that security methodology. These options are also impacted by other configuration settings of Record Center, such as your approval model—since determining who can approve records and at what point in their record lifecycle also contains an element of record security. Fundamentally, Record Center presents these options all in a way that is more intuitive than having to manually design what an overall repository architecture looks like—one of the challenges that B&R’s Managing Director, Mike Oryszak brushed on as part of his previous Keys to Designing and Managing Large Repositories blog post.

RecordApprovals1.png

Options for Record Security

Record Center offers three separate security models that may be configured to meet your organization’s individual needs. These record security models apply after a record has been loaded into the system and processed through any necessary approvals, so it’s important to note that the selection of a specific security model does not require a user to be a consumer of record content in order to participate in the upload/ingress of content, or to perform one or more of the record’s approval stages

Entity

The entity security model provides the least granularity for record security. This is often a good fit for small teams where the number of users consuming record content is low, the organization’s corporate structure is thin, or security doesn’t need to vary between individual record types. Every record added to Record Center is assigned to a legal entity, such as B&R Business Solutions, LLC. For organizations that only have one legal entity, this field will default to a single value, but from a security perspective this effectively means that all of that entity’s records are available to the same audience, be it every employee or a smaller subset such as a compliance team. This model is also particularly useful if an organization contains many different legal entities. This is often the case in the real estate industry where different properties are often separate legal entities. Using this model, records are easily classified and secured by the entity they belong to, simplifying the ability to grant users or owners of each entity access to only that entity’s records.

  The entity security model is a good fit for small teams or where security doesn’t need to vary between individual record types

Series

The series security model ensures that each individual record series created within Record Center can be individually secured. This allows you to provide granular access to specific categories of records, including all of the document types that belong to that specific series. As an example, providing a user access to a “Service Contracts” series, would give them access to all service contracts document types, which might include things like equipment leases, maintenance contracts, master service agreements, etc.

  Record Center’s Metadata-based security model allows for a more dynamic implementation of record security.

Metadata

Record Center’s Metadata-based security model allows for a more dynamic implementation of record security. When using this model, an organization defines one or more record metadata field(s) that can be used to determine that record’s security. As an example, if a record type has a “Business Unit” field, and the goal is to secure records based on if they were part of the Manufacturing business unit or Corporate business unit, metadata-based security would allow an organization to define users that will have access to any record where Business Unit is set to Manufacturing. This security is applied regardless of legal entity or record series, meaning that multiple metadata-based security fields may exist on any record type. Ultimately, this allows you to define the previous Business Unit based audiences in addition to say “Office Location”, where one or more users would be granted access to all records based on a specific Office Location value.

  Record Center’s Metadata-based security model allows for a more dynamic implementation of record security.

Compliance Access

In addition to the previously mentioned security models, Record Center also facilitates simple access for those users that need access to every record, such as a corporate compliance department. These users may be given access to Record Center’s “Global Record Access” group, which is applied to every record that is loaded into the system.

Conclusion

Our goal with Record Center has always been to try and simplify the otherwise daunting and complex task of designing and implementing a robust Records Management solution, be it the initial installation of a solution, designing the overall implementation, identifying an organization’s various record types, defining individual retention plans for each of those types, and ensuring that the right people can find the content they need when they need it. While Record Center’s ability to manage record security in a way that’s easy to understand is just one component of that strategy, it is vital to reducing accidental exposure, and ensuring that sensitive records are locked down to only those users that have been identified as consumers of that content.

About Record Center

Record Center is your turnkey solution for enterprise-class record management. An extension of Microsoft SharePoint, Record Center arms your users and record managers with a feature-packed, intuitive solution to manage the entire life-cycle of your records. Configure, Approve and Search for records faster and easier than ever with Record Center.

calltoaction-recordcenter.jpg

Interested in learning more about Record Center?

Overcoming Upload & Approval Challenges with Records Management

When we first embarked on the journey of creating Record Center, we knew there were several major pain points in the lifecycle of records management that we wanted to address. Creating a platform that would ease the burden on record managers and record approvers was one of our top priorities, and the challenges of content ingress and record approval were some of the main issues we wanted to address. 

To understand the pain of loading and approving content into a records management solution is to realize that content comes from a seemingly endless number of sources. Those sources could include other business systems or raw scans of physical documents, and could comprise any type of document an organization handles—contracts, invoices, employment agreements, non-disclosure agreements, tax documents, etc. Each of those unique document types could very well contain different metadata, different required fields, or different document formats further complicating the notion of a centralized ingress and approval process. 

Upload Content (Ingress) 

Record Center’s model of “Document Types” allows for an organization to define criteria for each type of record to be loaded into the system. Each of these document types contains its own distinct metadata requirements, approval requirements, retention plans, and disposition processes. This model allows for all records—regardless of their type—to be submitted to the same central Pending Records library, using the same process, while still ensuring that a unified record submission and approval flow may be used. 

Required Fields 

Critical to being able to easily and accurately find records is the notion of applying metadata to records. Record Center’s unique in-line preview mode allows a user submitting a record to easily see their document alongside the document’s required and optional metadata fields. This view allows them to quickly determine what fields must be completed for a given document type, and if necessary, navigate through the document in question to determine the values of those required fields. 

Approval Models 

Document Approvers and Record Managers are frequently faced with the challenge of having to approve large quantities of records. To cope with these challenges, Record Center offers several features integral to streamlining this otherwise time-consuming process. 

Unique Document Type Approvers

Each unique document type may contain its own distinct record approvers, allowing for a first stage approval by a user belonging to that document’s work center or area of expertise. This process helps to ensure that misclassified records or those with incorrect metadata are caught before entering the Record Manager’s approval queue. 

Inline Document Preview

Utilizing inline preview, a document approver and/or Record Manager can easily review a document alongside its metadata to ensure the metadata entered is accurate, and all required fields have been completed. 

Approve and View Next

As a document approver or Record Manager moves through their approval queue, they would historically have to open the document to be approved, approve that document, close it, open the next document, approve that, close it, and so on. To streamline this click-heavy process, Record Center contains a “Approve and View Next” feature, which allows the user to review, approve and open the next document in their queue with a single click. 

Bulk Approval

In the case of a bulk import where a record’s data has already been validated, Record Center includes a bulk approval option that allows a document approver or Record Manager to select multiple records at once and approve them all with a single click. 

Auto Approval

In some cases, an organization may wish to automatically approve records. The likely scenario for this is when a records management team is small, and perhaps the same people would do the content ingress as well as the approval. In this scenario, records can be configured to automatically approve after they’ve been pending approval for a set amount of time. 

 

About Record Center 

Record Center is your turnkey solution for enterprise-class record management. An extension of Microsoft SharePoint, Record Center arms your users and record managers with a feature-packed, intuitive solution to manage the entire life-cycle of your records. Configure, Approve and Search for records faster and easier than ever with Record Center. 

 
calltoaction-recordcenter.jpg

Interested in learning more about Record Center?

Finding Records with Record Center

Record Center offers users three distinct methods for finding records—each ensuring the security and integrity of the record by only showing a user those records they have been granted access to view. Users tend to mature through these three methods as they gain more understanding of the system and the structure or records within their organization.

 
icon-rc-folders-01.png

Traditional Folder-Like Browsing

“Browse Records” allows a user to navigate through the virtual hierarchy of records defined within the system. This method feels at home to those users that have historically worked in file shares or similar environments, or those that are new to the organization and may want to “explore” to better understand the record hierarchy.

 

Basic Keyword Searching

As a user learns more about the types of records stored within the system, record hierarchies, and metadata associated with given document types, they typically migrate away from browsing and begin to adopt search as the mechanism to find records. Record Center’s generic keyword searching allows you to perform simple keyword searches, or even create your own search queries to find the content you’re looking for.

A simple keyword search for “Lease” as an example, would return any record the user has access to that contains the word “Lease” in any of the record metadata, or in the content of the record itself (provided the record has gone through an OCR/indexing process). When performing a basic keyword search, a user will typically get a larger and less targeted result set. This result set can then be further refined using standardized metadata refiners—allowing the user to incrementally reduce that result set and hone in on their targeted records.

 

Structured Searching with Record Finder

Record Finder allows a user to conduct a structured search and identify a very targeted set of records based on specific criteria. This is the most efficient way for a user to find a record, provided they know at least some of the record’s metadata. Perhaps a user knows that they’re looking for a lease that was associated with a specific business unit, specific office, and affiliated with a specific legal entity. Record Finder allows a user to plug in all known metadata and issue a search for any matching records stored within Record Center. In addition, Record Finder can be configured to support different, allowing different users access to different searching scenarios with differing search fields. Furthermore, the targeted result set returned by Record Finder is may still be refined further using the standardized metadata refiners also available through the basic keyword searching.

 

About Record Center

Record Center is your turnkey solution for enterprise-class record management. An extension of Microsoft SharePoint, Record Center arms your users and record managers with a feature-packed, intuitive solution to manage the entire life-cycle of your records. Configure, Approve and Search for records faster and easier than ever with Record Center.

 

Interested in learning more about Record Center?

Keys to Designing and Managing Large Repositories

The B&R team has some deep experience managing large structured document repositories within SharePoint. In some cases, those repositories were established and grew within SharePoint organically over time, while in others there were large sets of content that were migrated in from networked file shares or other ECM solutions like Documentum, OpenText, or FileNet.

Throughout SharePoint’s long history there has often been confusion between software limitations and best practices. To make matters worse in many cases there is no global agreement on what the best practices are. In our experience, many of the best practices are really guidelines that are context specific. There is no generic answer, but rather a starting point from which the requirements can then shape the most appropriate solution within the right context. In our experience, some of those key decision points are:

Structured vs. Unstructured Content

paperclips-unorganized-sm.png

Loosely categorized unstructured content stored in desperate locations across the organization.

paperclips-organized-md.png

Centrally managed structured content that has been properly categorized.

While SharePoint can be used in many ways, the first contextual decision point is structured versus unstructured content. In this post, we will be specifically focused on structured content storage repositories with content types and meta-data defined, and not unstructured repositories. This is an important differentiator for us, since the organization and usability of content in unstructured systems is radically different.

Software Boundaries and IT Capabilities

Understanding the limits of your platforms and systems is vitally important.

When thinking of the actual software and storage boundaries, SharePoint as a platform is very flexible, and continues to increase limits as modern storage and database limits increase. Here are references to the software boundaries for 2013, 2016, and Office 365.

One of the common misconceptions is that the infamous “List View Threshold” implemented in SharePoint 2010 is a boundary limiting the number of items in a list or library to 5,000. This is not a limit to the number of items in a library, but rather the number of records that can be read as part of a database query. This specific topic will be addressed as part of the System User Experience and Content Findability section.

For on-premises versions of SharePoint Server including 2010, 2013, and 2016 our focus has been on establishing system sizes that can be reliably maintained including backup and recovery procedures. This is a pretty important point because in my experience the capabilities and expectations of my clients vary widely. In some cases, they have deep experience with multi-terabyte databases and have plenty of room to work with to both backup and restore databases as needed. In other cases some customers struggle with backing up and restoring databases that are just a few hundred gigabytes due to backup technologies or lack of available working storage space. With this in mind, our initial guiding points are to look at how to isolate the structured repositories into dedicated site collections, each with a dedicated content database. The number and size of those site collections vary depending on the customer’s requirements and backup and recovery capabilities. We frequently start by advising on smaller database sizes of around 100 GB and then adjust based on their comfort levels and capabilities, but they should never exceed the Sys Admin’s ability to capture and restore a database backup.

For Office 365, Microsoft has taken ownership of the system maintenance and regular backup and recovery operations. Within, the service they have also extended the software boundaries which can make it much easier to support systems with larger repositories and fewer site collections, pushing much of the decision to the next two points relating to system usability and content findability.

System User Experience and Content Findability

The user experience of the repository is essential to its long-term success.

We will focus on looking at the processes to initially add or author content and then how it will be later discovered. Patterns and techniques that work fine in other sites or repositories can completely fail with large repositories.

While SharePoint as a platform is typically thought of in terms of collaboration and collaborative content, the scenarios for structured content in large repositories is often different. In some scenarios, the content may be comprised of scanned documents or images that are sent directly to SharePoint, while in others they could be bulk loaded electronic documents.

Unlike the collaborative scenarios, you very rarely want to add the content to the root of a SharePoint library, but either organize the content across libraries and/or sub-folders. To better handle this scenario, we will often incorporate the Content Organizer feature that Microsoft made available with SharePoint Server 2010 which offers a temporary drop off library and rules to selectively route content to another site collection, library, or folder. This rules based approach provides great automation capabilities that help to keep things properly organized, while making it much easier to add content to the system. While the Content Organizer covers most of our common scenarios, we are able to support even more advanced scenarios for automation by leveraging a workflow tool or customization when needed.

Previously, the List View Threshold feature was mentioned. While it is often discussed as a boundary or limitation, it is actually a feature intended to help maintain system performance. For SharePoint Server 2010, 2013, and 2016 it is a system setting that can be set at the web application level. The intention of this feature is to provide protection against unoptimized queries being run against the back-end SQL server. The default value of 5,000 was chosen because that is the point in which queries are processed differently by the database’s query engine, and you will start to see performance related problems. While it is safe to make small changes beyond the default limit, quickly you will experience the performance impacts the feature was designed to avoid.

The important thing to remember is that the threshold is for a given query, so the key task is to plan and implement your views to be optimized. We do this by thinking about a few key things:

Configure the Default View:

By default, SharePoint points to the All Items for the default view. Ideally, no view will be without a viable filter, but the All Items view absolutely should not be the default view in these libraries.

Column Indexes:

Key columns used to drive views or as the primary filter within your list can be indexed to improve performance. Additional information can be found here.

View Filters:

Ideally, all views will be sufficiently filtered to be below the List View Threshold of 5,000 items. This will keep view load time low.

Lookup Fields:

Avoid the use of lookup fields, as these lookup fields will require inefficient queries that perform tables scans to return content. Even smaller repositories of just a few hundred items can exceed the List View Threshold because of the query formatting.

Avoid Group By, Collapsed Option:

While the ability to group by your meta-data can be powerful, we typically instruct our clients to avoid using the option to collapse the Group By selections. The collapse option has some unexpected behavior that will result in additional database queries for each of the group by levels and values and disregard the item limits and paging configuration. It is possible to limit a view to say 30 items, but if you configure it to group by a value and collapse it by default, the first category could have 1,000 items and the system will query and load the full list, ignoring the 30-item limit. This can have severe performance implications, and is typically the primary culprit when we are asked to help troubleshoot page load performance in a specific repository.

While the ability to easily and effectively locate content has a big impact on the user experience of the system, I would argue that it is the most critical and therefore one that needs to be thought through when working within the SharePoint platform so I have broken the topic out into its own section.

If you think about SharePoint sites on a scale or continuum from the small team sites with a few libraries containing a handful of documents up to large, enterprise repositories with millions of documents it should be clear that how you find and interact with content on the two opposite ends of the spectrum needs to evolve. As systems grow larger, well beyond the minimalistic List View Threshold levels, the system needs to become more sophisticated and move away from manual browsing to content or unstructured search keyword queries to a more intelligent search driven experience.

While most systems include a properly configured search service, a much smaller percentage have it optimized in a way that it can leveraged to provide structured searches or dynamically surface relevant content. This optimization takes place at two levels; first within the search service itself, and then with customizations available in the system.

Within the Search Service we will work to identify meta-data key fields that should be established as managed properties for doing specific property searching, determine which fields need to be leveraged within a results screen for sorting and use within refinements. These changes allow us to execute more precise search queries and optimize the search results for our specific needs.

Within the site, we will then look to define the scenarios that people need to look for and find content to define structured search forms and result pages optimized for those scenarios. In some cases, they are generic to the content in the repository, while in others the scenarios are specific to a given task or role helping to simply things for specific business processes. By leveraging structured search pages, we can provide an improved user experience that dramatically reduces the time it takes to locate the relevant content as the initial search results are smaller, and then easily paired down through relevant search refiners. In addition, on common landing pages we will leverage the additional search driven web parts to highlight relevant, related, or new content as needed to support the usage scenarios.

Our Approach to Designing Record Center

As we set out to design and implement our Record Center product, we knew that it must scale to tens of millions of records both with regards to technical performance and from a user experience perspective. To accomplish this, we automated the setup and configuration process in ways to help optimize the solution for our specific purpose and use case.

While doing a product feature overview is outside the scope of this post, we are happy to report that our approach and techniques have been successfully adopted by our clients and that today the average repository size is in the hundreds of thousands of documents while still meeting performance, usability, and system maintenance goals.

Next Steps

I hope that this post provided a good overview of how to plan and maintain large repositories. It is a big topic with lots of nuances and techniques that are learned over time in the trenches. If your group is struggling with designing and managing large repositories and needs help, reach out and setup a consultation. We can either assist your team with advisement services, or help with the implementation of a robust system.

Can We Help?

Contact us today for a free consultation!

Why We Created Record Center

From time to time when I’m talking to prospective customer about Record Center, I get the question “What made you decide to build this?” It’s a great question, and the answer stems from a conversation I had with the CIO of one of our customers back in 2011. For this particular customer, we had recently implemented SharePoint Server 2010 and were in the process of migrating their Microsoft Office SharePoint Server 2007 sites and content up to 2010, while also planning for a variety of new SharePoint-based applications. When we looked across the portfolio of applications that we were going to build along with all of the other non-SharePoint-based systems they were maintaining, one thing became apparent – their records management story was non-existent. Across the organization, there were at least (that we were aware of) 11 different systems/locations that records could be stored in, and fewer than half of those had an identified business owner. To make matters worse, there was zero consistency in naming conventions, metadata, and classification schemas between the systems. It became immediately apparent that something had to be done because this was a mess from every perspective – but what exactly we were going to do was not so apparent.

You would have thought that with B&R being a SharePoint solutions provider, we would have immediately recommended SharePoint as the answer. But back in those days (and even still today), SharePoint was not known for being a great records management solution. Sure, there are site columns, managed metadata, content types, information management policies, retention policies, records declaration, and the content organizer (to name some of the features). But trying to explain how to configure and properly utilize all of these features to a site owner or business user tasked with records management for their department or group was close to impossible. The features were all over the place – library settings, site settings, site collection settings – and required various levels of access that most organizations did not want to grant. Simply put, SharePoint did not offer the end-to-end records management solution that was intuitive and easy to use; with this in mind we initially looked at alternative solutions.

brick-wall-construction.jpg

The features were all over the place and required various levels of access...

As an alternative to SharePoint, we looked at a variety of options including Documentum, FileNet, and other large systems, but the sticker shock we got when we saw not only the licensing but implementation costs (and timelines) always brought us back to SharePoint.

And that’s when Record Center was truly envisioned. When looking at the other systems, we saw the types of features and functionality they had – and we knew most of that was available in SharePoint, but was buried and difficult to implement out of the box. We knew what we had to do – take all of the disparate, but vitally important features of SharePoint that would support a records management initiative and combine them into one unified solution. And so, over the next two years, we built the application from the ground up, squeezing everything we could out of SharePoint while providing a simplified interface that provided records managers with the tools they needed to successfully perform their jobs and end users with the simplest of interfaces that allows them to find exactly what they are looking for without needing more than a few minutes of training.

In the end, Record Center became – and still is – one of the most successful IT projects in the history of our customer. And today, with the latest version of Record Center available for both SharePoint 2013 & 2016, organizations can have an accurate picture of and can easily manage the lifecycle of their records.

brick-wall-built.jpg

We knew we had to take all of the disparate features of SharePoint and combine them into one unified solution.

We know that records management isn’t a glamorous or exciting topic, but it’s a critical component to most organizational strategies for reducing liability, ensuring discoverability, and properly classifying records to meet legal and compliance requirements. Already using SharePoint? Then why implement a separate system that you must manage and maintain – use what you already have and experience a better path forward.

Interested in learning more about Record Center?