Programming | Tech News, Tutorials & Expert Insights

05 Mar 2015

8 min read

Advanced Cypher tricks

05 Mar 2015

Cypher is a highly efficient language that not only makes querying simpler but also strives to optimize the result-generation process to the maximum. A lot more optimization in performance can be achieved with the help of knowledge related to the data domain of the application being used to restructure queries. This article by Sonal Raj, the author of Neo4j High Performance, covers a few tricks that you can implement with Cypher for optimization. (For more resources related to this topic, see here.) Query optimizations There are certain techniques you can adopt in order to get the maximum performance out of your Cypher queries. Some of them are: Avoid global data scans: The manual mode of optimizing the performance of queries depends on the developer's effort to reduce the traversal domain and to make sure that only the essential data is obtained in results. A global scan searches the entire graph, which is fine for smaller graphs but not for large datasets. For example: START n =node(*) MATCH (n)-[:KNOWS]-(m) WHERE n.identity = "Batman" RETURN m Since Cypher is a greedy pattern-matching language, it avoids discrimination unless explicitly told to. Filtering data with a start point should be undertaken at the initial stages of execution to speed up the result-generation process. In Neo4j versions greater than 2.0, the START statement in the preceding query is not required, and unless otherwise specified, the entire graph is searched. The use of labels in the graphs and in queries can help to optimize the search process for the pattern. For example: START n =node(*) MATCH (n:superheroes)-[:KNOWS]-(m) WHERE n.identity = "Batman" RETURN m Using the superheroes label in the preceding query helps to shrink the domain, thereby making the operation faster. This is referred to as a label-based scan. Indexing and constraints for faster search: Searches in the graph space can be optimized and made faster if the data is indexed, or we apply some sort of constraint on it. In this way, the traversal avoids redundant matches and goes straight to the desired index location. To apply an index on a label, you can use the following: CREATE INDEX ON: superheroes(identity) Otherwise, to create a constraint on the particular property such as making the value of the property unique so that it can be directly referenced, we can use the following: CREATE CONSTRAINT ON n:superheroes ASSERT n.identity IS UNIQUE We will learn more about indexing, its types, and its utilities in making Neo4j more efficient for large dataset-based operations in the next sections. Avoid Cartesian Products Generation: When creating queries, we should include entities that are connected in some way. The use of unspecific or nonrelated entities can end up generating a lot of unused or unintended results. For example: MATCH (m:Game), (p:Player) This will end up mapping all possible games with all possible players and that can lead to undesired results. Let's use an example to see how to avoid Cartesian products in queries: MATCH ( a:Actor), (m:Movie), (s:Series) RETURN COUNT(DISTINCT a), COUNT(DISTINCT m), COUNT(DISTINCTs) This statement will find all possible triplets of the Actor, Movie, and Series labels and then filter the results. An optimized form of querying will include successive counting to get a final result as follows: MATCH (a:Actor) WITH COUNT(a) as actors MATCH (m:Movie) WITH COUNT(m) as movies, actors MATCH (s:Series) RETURN COUNT(s) as series, movies, actors This increases the 10x improvement in the execution time of this query on the same dataset. Use more patterns in MATCH rather than WHERE: It is advisable to keep most of the patterns used in the MATCH clause. The WHERE clause is not exactly meant for pattern matching; rather it is used to filter the results when used with START and WITH. However, when used with MATCH, it implements constraints to the patterns described. Thus, the pattern matching is faster when you use the pattern with the MATCH section. After finding starting points—either by using scans, indexes, or already-bound points—the execution engine will use pattern matching to find matching subgraphs. As Cypher is declarative, it can change the order of these operations. Predicates in WHERE clauses can be evaluated before, during, or after pattern matching. Split MATCH patterns further: Rather than having multiple match patterns in the same MATCH statement in a comma-separated fashion, you can split the patterns in several distinct MATCH statements. This process considerably decreases the query time since it can now search on smaller or reduced datasets at each successive match stage. When splitting the MATCH statements, you must keep in mind that the best practices include keeping the pattern with labels of the smallest cardinality at the head of the statement. You must also try to keep those patterns generating smaller intermediate result sets at the beginning of the match statements block. Profiling of queries: You can monitor your queries' processing details in the profile of the response that you can achieve with the PROFILE keyword, or setting profile parameter to True while making the request. Some useful information can be in the form of _db_hits that show you how many times an entity (node, relationship, or property) has been encountered. Returning data in a Cypher response has substantial overhead. So, you should strive to restrict returning complete nodes or relationships wherever possible and instead, simply return the desired properties or values computed from the properties. Parameters in queries: The execution engine of Cypher tries to optimize and transform queries into relevant execution plans. In order to optimize the amount of resources dedicated to this task, the use of parameters as compared to literals is preferred. With this technique, Cypher can re-utilize the existing queries rather than parsing or compiling the literal-hbased queries to build fresh execution plans: MATCH (p:Player) –[:PLAYED]-(game) WHERE p.id = {pid} RETURN game When Cypher is building execution plans, it looks at the schema to see whether it can find useful indexes. These index decisions are only valid until the schema changes, so adding or removing indexes leads to the execution plan cache being flushed. Add the direction arrowhead in cases where the graph is to be queries in a directed manner. This will reduce a lot of redundant operations. Graph model optimizations Sometimes, the query optimizations can be a great way to improve the performance of the application using Neo4j, but you can incorporate some fundamental practices while you define your database so that it can make things easier and faster for usage: Explicit definition: If the graph model we are working upon contains implicit relationships between components. A higher efficiency in queries can be achieved when we define these relations in an explicit manner. This leads to faster comparisons but it comes with a drawback that now the graph would require more storage space for an additional entity for all occurrences of data. Let's see this in action with the help of an example. In the following diagram, we see that when two players have played in the same game, they are most likely to know each other. So, instead of going through the game entity for every pair of connected players, we can define the KNOWS relationship explicitly between the players. Property refactoring: This refers to the situation where complex time-consuming operations in the WHERE or MATCH clause can be included directly as properties in the nodes of the graph. This not only saves computation time resulting in much faster queries but it also leads to more organized data storage practices in the graph database for utility. For example: MATCH (m:Movie) WHERE m.releaseDate >1343779201 AND m.releaseDate< 1369094401 RETURN m This query is to compare whether a movie has been released in a particular year; it can be optimized if the release year of the movie is inherently stored in the properties of the movie nodes in the graph as the year range 2012-2013. So, for the new format of the data, the query will now change to this: MATCH (m:Movie)-[:CONTAINS]->(d) WHERE s.name = "2012-2013" RETURN g This gives a marked improvement in the performance of the query in terms of its execution time. Summary These are the various tricks that can be implemented in Cypher for optimization. Resources for Article: Further resources on this subject: Recommender systems dissected [Article] Working with a Neo4j Embedded Database [Article] Adding Graphics to the Map [Article]

0
0
8190

article-image-getting-started-codeblocks

Packt

28 Oct 2013

7 min read

Getting Started with Code::Blocks

Packt

28 Oct 2013

7 min read

0
0
8147

article-image-documenting-your-python-project-part1

Packt

12 Oct 2009

7 min read

Documenting Your Python Project-part1

Packt

12 Oct 2009

7 min read

Documenting Your Project Documentation is work that is often neglected by developers and sometimes by managers. This is often due to a lack of time towards the end of development cycles, and the fact that people think they are bad at writing. Some of them are bad, but the majority of them are able to produce fine documentation. In any case, the result is a disorganized documentation made of documents that are written in a rush. Developers hate doing this kind of work most of the time. Things get even worse when existing documents need to be updated. Many projects out there are just providing poor, out-of-date documentation because the manager does not know how to deal with it. But setting up a documentation process at the beginning of the project and treating documents as if they were modules of code makes documenting easier. Writing can even be fun when a few rules are followed. This article provides a few tips to start documenting your project through: The seven rules of technical writing that summarize the best practices A reStructuredText primer, which is a plain text markup syntax used in most Python projects A guide for building good project documentation The Seven Rules of Technical Writing Writing good documentation is easier in many aspects than writing a code. Most developers think it is very hard, but by following a simple set of rules it becomes really easy. We are not talking here about writing a book of poems but a comprehensive piece of text that can be used to understand a design, an API, or anything that makes up the code base. Every developer is able to produce such material, and this section provides seven rules that can be applied in all cases. Write in two steps: Focus on ideas, and then on reviewing and shaping your text. Target the readership: Who is going to read it? Use a simple style: Keep it straight and simple. Use good grammar. Limit the scope of the information: Introduce one concept at a time. Use realistic code examples: Foos and bars should be dropped. Use a light but sufficient approach: You are not writing a book! Use templates: Help the readers to get habits. These rules are mostly inspired and adapted from Agile Documenting, a book by Andreas Rüping that focuses on producing the best documentation in software projects. Write in Two Steps Peter Elbow, in Writing with Power, explains that it is almost impossible for any human being to produce a perfect text in one shot. The problem is that many developers write documentation and try to directly come up with a perfect text. The only way they succeed in this exercise is by stopping the writing after every two sentences to read them back, and do some corrections. This means that they are focusing both on the content and the style of the text. This is too hard for the brain and the result is often not as good as it could be. A lot of time and energy is spent in polishing the style and shape of the text, before its meaning is completely thought through. Another approach is to drop the style and organization of the text and focus on its content. All ideas are laid down on paper, no matter how they are written. The developer starts to write a continuous stream and does not pause when he or she makes grammatical mistakes, or for anything that is not about the content. For instance, it does not matter if the sentences are barely understandable as long as the ideas are written down. He or she just writes down what he wants to say, with a rough organization. By doing this, the developer focuses on what he or she wants to say and will probably get more content out of his or her brain than he or she initially thought he or she would. Another side-effect when doing free writing is that other ideas that are not directly related to the topic will easily go through the mind. A good practice is to write them down on a second paper or screen when they appear, so they are not lost, and then get back to the main writing. The second step consists of reading back the whole text and polishing it so that it is comprehensible to everyone. Polishing a text means enhancing its style, correcting its faults, reorganizing it a bit, and removing any redundant information it has. When the time dedicated to write documentation is limited, a good practice is to cut this time in two equal durations—one for writing the content, and one to clean and organize the text. Focus on the content, and then on style and cleanliness. Target the Readership When starting a text, there is a simple question the writer should consider: Who is going to read it? This is not always obvious, as a technical text explains how a piece of software works, and is often written for every person who might get and use the code. The reader can be a manager who is looking for an appropriate technical solution to a problem, or a developer who needs to implement a feature with it. A designer might also read it to know if the package fits his or her needs from an architectural point of view. Let's apply a simple rule: Each text should have only one kind of readers. This philosophy makes the writing easier. The writer precisely knows what kind of reader he or she is dealing with. He or she can provide a concise and precise documentation that is not vaguely intended for all kinds of readers. A good practice is to provide a small introductory text that explains in one sentence what the documentation is about, and guides the reader to the appropriate part: Atomisator is a product that fetches RSS feeds and saves them in adatabase, with a filtering process.If you are a developer, you might want to look at the API description(api.txt)If you are a manager, you can read the features list and the FAQ(features.txt)If you are a designer, you can read the architecture andinfrastructure notes (arch.txt) By taking care of directing your readers in this way, you will probably produce better documentation. Know your readership before you start to write. Use a Simple Style Seth Godin is one of the best-selling writers on marketing topics. You might want to read Unleashing the Ideavirus, which is available for free on the Internet http://en.wikipedia.org/wiki/Unleashing_the_Ideavirus. Lately, he made an analysis on his blog to try to understand why his books sold so well. He made a list of all best sellers in the marketing area and compared the average number of words per sentences in each one of them. He realized that his books had the lowest number of words per sentence (thirteen words). This simple fact, Seth explained, proved that readers prefer short and simple sentences, rather than long and stylish ones. By keeping sentences short and simple, your writings will consume less brain power for their content to be extracted, processed, and then understood. Writing technical documentation aims to provide a software guide to readers. It is not a fiction story, and should be closer to your microwave notice than to the latest Stephen King novel. A few tips to keep in mind are: Use simple sentences; they should not be longer than two lines. Each paragraph should be composed of three or four sentences, at the most, that express one main idea. Let your text breathe. Don't repeat yourself too much: Avoid journalistic styles where ideas are repeated again and again to make sure they are understood. Don't use several tenses. Present tense is enough most of the time. Do not make jokes in the text if you are not a really fine writer. Being funny in a technical book is really hard, and few writers master it. If you really want to distill some humor, keep it in code examples and you will be fine. You are not writing fiction, so keep the style as simple as possible.

0
0
8137

article-image-auto-updating-child-records-process-builder

Packt

29 Apr 2015

5 min read

Auto updating child records in Process Builder

Packt

29 Apr 2015

5 min read

In this article by Rakesh Gupta, the author of the book Learning Salesforce Visual Workflow, we will discuss how to auto update child records using Process Builder of Salesforce. There are several business use cases where a customer wants to update child records based on some criteria, for example, auto-updating all related Opportunity to Closed-Lost if an account is updated to Inactive. To achieve these types of business requirements, you can use the Apex trigger. You can also achieve these types of requirements using the following methods: Process Builder A combination of Flow and Process Builder A combination of Flow and Inline Visualforce page on the account detail page (For more resources related to this topic, see here.) We will use Process Builder to solve these types of business requirements. Let's start with a business requirement. Here is a business scenario: Alice Atwood is working as a system administrator in Universal Container. She has received a requirement that once an account gets activated, the account phone must be synced with the related contact asst. phone field. This means whenever an account phone fields gets updated, the same phone number will be copied to the related contacts asst. phone field. Follow these instructions to achieve the preceding requirement using Process Builder: First of all, navigate to Setup | Build | Customize | Accounts | Fields and make sure that the Active picklist is available in your Salesforce organization. If it's not available, create a custom Picklist field with the name as Active, and enter the Yes and No values. To create a Process, navigate to Setup | Build | Create | Workflow & Approvals | Process Builder, click on New Button, and enter the following details: Name: Enter the name of the Process. Enter Update Contacts Asst Phone in Name. This must be within 255 characters. API Name: This will be autopopulated based on the name. Description: Write some meaningful text so that other developers or administrators can easily understand why this Process is created. The properties window will appear as shown in the following screenshot: Once you are done, click on the Save button. It will redirect you to the Process canvas, which allows you to create or modify the Process. After Define Process Properties, the next task is to select the object on which you want to create a Process and define the evaluation criteria. For this, click on the Add Object node. It will open an additional window on the right side of the Process canvas screen, where you have to enter the following details: Object: Start typing and then select the Account object. Start the process: For Start the process, select when a record is created or edited. This means the Process will fire every time, irrespective of record creation or updating. Allow process to evaluate a record multiple times in a single transaction?: Select this checkbox only when you want the Process to evaluate the same record up to five times in a single transaction. It might re-examine the record because a Process, Workflow Rule, or Flow may have updated the record in the same transaction. In this case, leave this unchecked. This window will appear as shown in the following screenshot: Once you are done with adding the Process criteria, click on the Save button. Similar to the Workflow Rule, once you save the panel, it doesn't allow you to change the selected object. After defining the evaluation criteria, the next step is to add the Process criteria. Once the Process criteria are true, only then will the Process execute the associated actions. To define the Process criteria, click on the Add Criteria node. It will open an additional window on the right side of the Process canvas screen, where you have to enter the following details: Criteria Name: Enter a name for the criteria node. Enter Update Contacts in Criteria Name. Criteria for Executing Actions: Select the type of criteria you want to define. You can use either a formula or a filter to define the Process criteria or no criteria. In this case, select Active equals to Yes. This means the Process will fire only when the account is active. This window will appear as shown in the following screenshot: Once you are done with defining the Process criteria, click on the Save button. Once you are done with the Process criteria node, the next step is to add an immediate action to update the related contact's asst. phone field. For this, we will use the Update Records action available under Process. Click on Add Action available under IMMEDIATE ACTIONS. It will open an additional window on the right side of the Process canvas screen, where you have to enter the following details: Action Type: Select the type of action. In this case, select Update Records. Action Name: Enter a name for this action. Enter Update Assts Phone in Action Name. Object: Start typing and then select the [Account].Contacts object. Field: Map the Asst. Phone field with the [Account]. Phone field. To select the fields, you can use field picker. To enter the value, use the text entry field. It will appear as shown in the following screenshot: Once you are done, click on the Save button. Once you are done with the immediate action, the final step is to activate it. To activate a Process, click on the Activate button available on the button bar. From now on, if you try to update an active account, Process will automatically update the related contact's asst. phone with the value available in the account phone field. Summary In this article, we have learned the technique of auto updating records in Process Builder. Resources for Article: Further resources on this subject: Visualforce Development with Apex [Article] Configuration in Salesforce CRM [Article] Introducing Salesforce Chatter [Article]

0
0
8135

article-image-python-multimedia-video-format-conversion-manipulations-and-effects

Packt

10 Dec 2010

11 min read

Python Multimedia: Video Format Conversion, Manipulations and Effects

Packt

10 Dec 2010

11 min read

0
0
8114

Packt

27 Aug 2013

3 min read

Salesforce CRM Functions

Packt

27 Aug 2013

3 min read

(For more resources related to this topic, see here.) Functional overview of Salesforce CRM The Salesforce CRM functions are related to each other and, as mentioned previously, have cross-over areas which can be represented as shown in the following diagram: Marketing administration Marketing administration is available in Salesforce CRM under the application suite known as the Marketing Cloud. The core functionality enables organizations to manage marketing campaigns from initiation to lead development in conjunction with the sales team. The features in the marketing application can help measure the effectiveness of each campaign by analyzing the leads and opportunities generated as a result of specific marketing activities. Salesforce automation Salesforce automation is the core feature set within Salesforce CRM and is used to manage the sales process and activities. It enables salespeople to automate manual and repetitive tasks and provides them with information related to existing and prospective customers. In Salesforce CRM, Salesforce automation is known as the Sales Cloud, and helps the sales people manage sales activities, leads and contact records, opportunities, quotes, and forecasts. Customer service and support automation Customer service and support automation within Salesforce CRM is known as the Service Cloud, and allows support teams to automate and manage the requests for service and support by existing customers. Using the Service Cloud features, organizations can handle customer requests, such as the return of faulty goods or repairs, complaints, or provide advice about products and services. Associated with the functional areas, described previously, are features and mechanisms to help users and customers collaborate and share information known as enterprise social networking. Enterprise social networking Enterprise social network capabilities within Salesforce CRM enable organizations to connect with people and securely share business information in real time. Social networking within an enterprise serves to connect both employees and customers and enables business collaboration. In Salesforce CRM, the enterprise social network suite is known as Salesforce Chatter. Salesforce CRM record life cycle The capabilities of Salesforce CRM provides for the processing of campaigns through to customer acquisition and beyond as shown in the following diagram: At the start of the process, it is the responsibility of the marketing team to develop suitable campaigns in order to generate leads. Campaign management is carried out using the Marketing Administration tools and has links to the lead and also any opportunities that have been influenced by the campaign. When validated, leads are converted to accounts, contacts, and opportunities. This can be the responsibility of either the marketing or sales teams and requires a suitable sales process to have been agreed upon. In Salesforce CRM, an account is the company or organization and a contact is an individual associated with an account. Opportunities can either be generated from lead conversion or may be entered directly by the sales team. As described earlier in this article, the structure of Salesforce requires account ownership to be established which sees inherited ownership of the opportunity. Account ownership is usually the responsibility of the sales team. Opportunities are worked through a sales process using sales stages where the stage is advanced to the point where they are set as won/closed and become sales. Opportunity information should be logged in the organization's financial system. Upon financial completion and acceptance of the deal (and perhaps delivery of the goods or service), the post-customer acquisition process is then enabled where the account and contact can be recognized as a customer. Here the customer relationships concerning incidents and requests are managed by escalating cases within the customer services and support automation suite.

0
0
8101

article-image-python-libraries-geospatial-development

Packt

17 Jun 2013

14 min read

Python Libraries for Geospatial Development

Packt

17 Jun 2013

14 min read

(For more resources related to this topic, see here.) Reading and writing geospatial data While you could in theory write your own parser to read a particular geospatial data format, it is much easier to use an existing Python library to do this. We will look at two popular libraries for reading and writing geospatial data: GDAL and OGR. GDAL/OGR Unfortunately, the naming of these two libraries is rather confusing. Geospatial Data Abstraction Library ( GDAL), was originally just a library for working with raster geospatial data, while the separate OGR library was intended to work with vector data. However, the two libraries are now partially merged, and are generally downloaded and installed together under the combined name of "GDAL". To avoid confusion, we will call this combined library GDAL/OGR and use "GDAL" to refer to just the raster translation library. A default installation of GDAL supports reading 116 different raster file formats, and writing to 58 different formats. OGR by default supports reading 56 different vector file formats, and writing to 30 formats. This makes GDAL/OGR one of the most powerful geospatial data translators available, and certainly the most useful freely-available library for reading and writing geospatial data. GDAL design GDAL uses the following data model for describing raster geospatial data: Let's take a look at the various parts of this model: A dataset holds all the raster data, in the form of a collection of raster "bands", along with information that is common to all these bands. A dataset normally represents the contents of a single file. A raster band represents a band, channel, or layer within the image. For example, RGB image data would normally have separate bands for the red, green, and blue components of the image. The raster size specifies the overall width and height of the image, in pixels. The georeferencing transform converts from (x, y) raster coordinates into georeferenced coordinates—that is, coordinates on the surface of the earth. There are two types of georeferencing transforms supported by GDAL: affine transformations and ground control points. An affine transformation is a mathematical formula allowing the following operations to be applied to the raster data: More than one of these operations can be applied at once; this allows you to perform sophisticated transforms such as rotations. Affine transformations are sometimes referred to as linear transformations. Ground Control Points ( GCPs) relate one or more positions within the raster to their equivalent georeferenced coordinates, as shown in the following figure: Note that GDAL does not translate coordinates using GCPs— that is left up to the application, and generally involves complex mathematical functions to perform the transformation. The coordinate system describes the georeferenced coordinates produced the georeferencing transform. The coordinate system includes the projection and datum, as well as the units and scale used by the raster data. The metadata contains additional information about the dataset as a whole. Each raster band contains the following (among other things): The band raster size: This is the size (number of pixels across and number of lines high) for the data within the band. This may be the same as the raster size for the overall dataset, in which case the dataset is at full resolution, or the band's data may need to be scaled to match the dataset. Some band metadata providing extra information specific to this band. A color table describing how pixel values are translated into colors. The raster data itself. GDAL provides a number of drivers which allow you to read (and sometimes write) various types of raster geospatial data. When reading a file, GDAL selects a suitable driver automatically based on the type of data; when writing, you first select the driver and then tell the driver to create the new dataset you want to write to. GDAL example code A Digital Elevation Model ( DEM) file contains height values. In the following example program, we use GDAL to calculate the average of the height values contained in a sample DEM file. In this case, we use a DEM file downloaded from the GLOBE elevation dataset: from osgeo import gdal,gdalconst import struct dataset = gdal.Open("data/e10g") band = dataset.GetRasterBand(1) fmt = "<" + ("h" * band.XSize) totHeight = 0 for y in range(band.YSize): scanline = band.ReadRaster(0, y, band.XSize, 1, band.XSize, 1, band.DataType) values = struct.unpack(fmt, scanline) for value in values: if value == -500: # Special height value for the sea -> ignore. continue totHeight = totHeight + value average = totHeight / (band.XSize * band.YSize) print "Average height =", average As you can see, this program obtains the single raster band from the DEM file, and then reads through it one scanline at a time. We then use the struct standard Python library module to read the individual height values out of the scanline. Because the GLOBE dataset uses a special height value of -500 to represent the ocean, we exclude these values from our calculations. Finally, we use the remaining height values to calculate the average height, in meters, over the entire DEM data file. OGR design OGR uses the following model for working with vector-based geospatial data: Let's take a look at this design in more detail: The data source represents the file you are working with—though it doesn't have to be a file. It could just as easily be a URL or some other source of data. The data source has one or more layers , representing sets of related data. For example, a single data source representing a country may contain a "terrain" layer, a "contour lines" layer, a "roads" later, and a "city boundaries" layer. Other data sources may consist of just one layer. Each layer has a spatial reference and a list of features. The spatial reference specifies the projection and datum used by the layer's data. A feature corresponds to some significant element within the layer. For example, a feature might represent a state, a city, a road, an island, and so on. Each feature has a list of attributes and a geometry. The attributes provide additional meta-information about the feature. For example, an attribute might provide the name for a city's feature, its population, or the feature's unique ID used to retrieve additional information about the feature from an external database. Finally, the geometry describes the physical shape or location of the feature. Geometries are recursive data structures that can themselves contain sub-geometries—for example, a "country" feature might consist of a geometry that encompasses several islands, each represented by a subgeometry within the main "country" geometry. The geometry design within OGR is based on the Open Geospatial Consortium's "Simple Features" model for representing geospatial geometries. For more information, see http://www.opengeospatial.org/standards/sfa . Like GDAL, OGR also provides a number of drivers which allow you to read (and sometimes write) various types of vector-based geospatial data. When reading a file, OGR selects a suitable driver automatically; when writing, you first select the driver and then tell the driver to create the new data source to write to. OGR example code The following example program uses OGR to read through the contents of a shapefile, printing out the value of the NAME attribute for each feature along with the geometry type: from osgeo import ogr shapefile = ogr.Open("TM_WORLD_BORDERS-0.3.shp") layer = shapefile.GetLayer(0) for i in range(layer.GetFeatureCount()): feature = layer.GetFeature(i) name = feature.GetField("NAME") geometry = feature.GetGeometryRef() print i, name, geometry.GetGeometryName() Documentation GDAL and OGR are well documented, but with a catch for Python programmers. The GDAL/OGR library and associated command-line tools are all written in C and C++. Bindings are available which allow access from a variety of other languages, including Python, but the documentation is all written for the C++ version of the libraries. This can make reading the documentation rather challenging—not only are all the method signatures written in C++, but the Python bindings have changed many of the method and class names to make them more "pythonic". Fortunately, the Python libraries are largely self-documenting, thanks to all the docstrings embedded in the Python bindings themselves. This means you can explore the documentation using tools such as Python's built-in pydoc utility, which can be run from the command line like this: % pydoc -g osgeo This will open up a GUI window allowing you to read the documentation using a web browser. Alternatively, if you want to find out about a single method or class, you can use Python's built-in help() command from the Python command line, like this: >>> import osgeo.ogr >>> help(osgeo.ogr.DataSource.CopyLayer) Not all the methods are documented, so you may need to refer to the C++ docs on the GDAL website for more information, and some of the docstrings are copied directly from the C++ documentation—but in general the documentation for GDAL/OGR is excellent, and should allow you to quickly come up to speed using this library. Availability GDAL/OGR runs on modern Unix machines, including Linux and Mac OS X, as well as most versions of Microsoft Windows. The main website for GDAL can be found at: http://gdal.org The main website for OGR is at: http://gdal.org/ogr To download GDAL/OGR, follow the Downloads link on the main GDAL website. Windows users may find the FWTools package useful, as it provides a wide range of geospatial software for win32 machines, including GDAL/OGR and its Python bindings. FWTools can be found at: http://fwtools.maptools.org For those running Mac OS X, prebuilt binaries can be obtained from: http://www.kyngchaos.com/software/frameworks Make sure that you install GDAL Version 1.9 or later, as you will need this version to work through the examples in this book. Being an open source package, the complete source code for GDAL/OGR is available from the website, so you can compile it yourself. Most people, however, will simply want to use a prebuilt binary version. Dealing with projections One of the challenges of working with geospatial data is that geodetic locations (points on the Earth's surface) are mapped into a two-dimensional Cartesian plane using a cartographic projection. Whenever you have some geospatial data, you need to know which projection that data uses. You also need to know the datum (model of the Earth's shape) assumed by the data. A common challenge when dealing with geospatial data is that you have to convert data from one projection/datum to another. Fortunately, there is a Python library pyproj which makes this task easy. pyproj pyproj is a Python "wrapper" around another library called PROJ.4. "PROJ.4" is an abbreviation for Version 4 of the PROJ library. PROJ was originally written by the US Geological Survey for dealing with map projections, and has been widely used in geospatial software for many years. The pyproj library makes it possible to access the functionality of PROJ.4 from within your Python programs. Design The pyproj library consists of the following pieces: pyproj consists of just two classes: Proj and Geod. Proj converts from longitude and latitude values to native map (x, y) coordinates, and vice versa. Geod performs various Great Circle distance and angle calculations. Both are built on top of the PROJ.4 library. Let's take a closer look at these two classes. Proj Proj is a cartographic transformation class, allowing you to convert geographic coordinates (that is, latitude and longitude values) into cartographic coordinates (x, y values, by default in meters) and vice versa. When you create a new Proj instance, you specify the projection, datum, and other values used to describe how the projection is to be done. For example, to use the Transverse Mercator projection and the WGS84 ellipsoid, you would do the following: projection = pyproj.Proj(proj='tmerc', ellps='WGS84') Once you have created a Proj instance, you can use it to convert a latitude and longitude to an (x, y) coordinate using the given projection. You can also use it to do an inverse projection—that is, converting from an (x, y) coordinate back into a latitude and longitude value again. The helpful transform() function can be used to directly convert coordinates from one projection to another. You simply provide the starting coordinates, the Proj object that describes the starting coordinates' projection, and the desired ending projection. This can be very useful when converting coordinates, either singly or en masse. Geod Geod is a geodetic computation class, which allows you to perform various Great Circle calculations. We looked at Great Circle calculations earlier, when considering how to accurately calculate the distance between two points on the Earth's surface. The Geod class, however, can do more than this: The fwd() method takes a starting point, an azimuth (angular direction) and a distance, and returns the ending point and the back azimuth (the angle from the end point back to the start point again): The inv() method takes two coordinates and returns the forward and back azimuth as well as the distance between them: The npts() method calculates the coordinates of a number of points spaced equidistantly along a geodesic line running from the start to the end point: When you create a new Geod object, you specify the ellipsoid to use when performing the geodetic calculations. The ellipsoid can be selected from a number of predefined ellipsoids, or you can enter the parameters for the ellipsoid (equatorial radius, polar radius, and so on) directly. Example code The following example starts with a location specified using UTM zone 17 coordinates. Using two Proj objects to define the UTM Zone 17 and lat/long projections, it translates this location's coordinates into latitude and longitude values: import pyproj UTM_X = 565718.5235 UTM_Y = 3980998.9244 srcProj = pyproj.Proj(proj="utm", zone="11", ellps="clrk66", units="m") dstProj = pyproj.Proj(proj="longlat", ellps="WGS84", datum="WGS84") long,lat = pyproj.transform(srcProj, dstProj, UTM_X, UTM_Y) print "UTM zone 11 coordinate (%0.4f, %0.4f) = %0.4f, %0.4f" % (UTM_X, UTM_Y, lat, long) Continuing on with this example, let's take the calculated lat/long values and, using a Geod object, calculate another point 10 kilometers northeast of that location: angle = 315 # 315 degrees = northeast. distance = 10000 geod = pyproj.Geod(ellps="WGS84") long2,lat2,invAngle = geod.fwd(long, lat, angle, distance) print "%0.4f, %0.4f is 10km northeast of %0.4f, %0.4f" % (lat2, long2, lat, long) Documentation The documentation available on the pyproj website, and in the docs directory provided with the source code, is excellent as far as it goes. It describes how to use the various classes and methods, what they do and what parameters are required. However, the documentation is rather sparse when it comes to the parameters used when creating a new Proj object. As the documentation says: A Proj class instance is initialized with proj map projection control parameter key/value pairs. The key/value pairs can either be passed in a dictionary, or as keyword arguments, or as a proj4 string (compatible with the proj command). The documentation does provide a link to a website listing a number of standard map projections and their associated parameters, but understanding what these parameters mean generally requires you to delve into the PROJ documentation itself. The documentation for PROJ is dense and confusing, even more so because the main manual is written for PROJ Version 3, with addendums for later versions. Attempting to make sense of all this can be quite challenging. Fortunately, in most cases you won't need to refer to the PROJ documentation at all. When working with geospatial data using GDAL or OGR, you can easily extract the projection as a "proj4 string" which can be passed directly to the Proj initializer. If you want to hardwire the projection, you can generally choose a projection and ellipsoid using the proj="..." and ellps="..." parameters, respectively. If you want to do more than this, though, you will need to refer to the PROJ documentation for more details. To find out more about PROJ, and to read the original documentation, you can find everything you need at: http://trac.osgeo.org/proj

0
0
8088

Packt

30 Jul 2013

19 min read

The Command Line

Packt

30 Jul 2013

19 min read

0
0
8043

Packt

20 Oct 2015

9 min read

Application Patterns

Packt

20 Oct 2015

9 min read

0
0
8030

article-image-tips-and-tricks-effectively-using-aspnet

Packt

15 Dec 2010

4 min read

Tips and Tricks for Effectively Using ASP.NET

Packt

15 Dec 2010

4 min read

ASP.NET Site Performance Secrets Simple and proven techniques to quickly speed up your ASP.NET website Speed up your ASP.NET website by identifying performance bottlenecks that hold back your site's performance and fixing them Tips and tricks for writing faster code and pinpointing those areas in the code that matter most, thus saving time and energy Drastically reduce page load times Configure and improve compression – the single most important way to improve your site's performance Written in a simple problem-solving manner – with a practical hands-on approach and just the right amount of theory you need to make sense of it all It is good to monitor the performance of your site continuously Tip: The performance of your website is affected by both the things you control, such as code changes, and the things you cannot control such as increases in the number of visitors or server problems. Because of this, it makes sense to monitor the performance of your site continuously. That way, you find out that the site is becoming too slow before your manager does. It is important to reduce the "time to first byte" Tip: The "time to first byte" is the time it takes your server to generate a page, plus the time taken to move the first byte over the Internet to the browser. Reducing that time is important for visitor retention—you want to give them something to look at, and provide confidence that they'll have the page in their browser soon. It involves making better use of system resources such as memory and CPU. Caching is one of the methods used to improve website performance Tip: Caching allows you to store individual objects, parts of web pages, or entire web pages in memory either in the browser, a proxy, or the server. That way, those objects or pages do not have to be generated again for each request, giving you: Reduced response time Reduced memory and CPU usage Less load on the database Fewer round trips to the server, when using browser or proxy caching Reduced retrieval times when the content is served from proxy cache, by bringing the contents closer to the browser Building projects in the Release mode is a good practice Tip: If your site is a web-application project rather than a website, or if your website is a part of a solution containing other projects, be sure to build your releases in the release mode. This removes debugging overhead from your code, so it uses less CPU and memory. It is important to reduce Round Trips between browser and server Tip: Round trips between browser and server can take a long time, increasing wait times for the visitor. Hence it is necessary to cut down on them. The same goes for round trips between web server and database server. Permanent redirects Tip: If you are redirecting visitors to a new page because the page is outdated, use a permanent 301 redirect. Browsers and proxies will update their caches, and search engines will use them as well. That way, you reduce traffic to the old page. You can issue a 301 redirect programmatically: Response.StatusCode = 301; Response.AddHeader("Location", "NewPage.aspx"); Response.End(); For .NET 4 or higher: Response.RedirectPermanent("NewPage.aspx"); Hotlinking should be avoided Tip: Hotlinking is the practice of linking to someone else's images on your site. If this happens to you and your images, another web master gets to show your images on their site and you get to pay for the additional bandwidth and incur the additional load on your server. A great little module that prevents hot linking is LeechGuard Hot-Linking Prevention Module at http://www.iis.net/community/default.aspx?tabid=34&i=1288&g=6. Session state is taking too much memory Tip: If you decide that session state is taking too much memory, here are some solutions. Reduce session state life time Reduce space taken by session state Use another session mode Stop using session state Cookies Tip: ASP.NET disables all output caching if you set a cookie, to make sure the cookie isn't sent to the wrong visitor. Since setting cookies and proxy caching also don't go together performance-wise, you'll want to keep setting the number of cookies to a minimum. This can be done by trying not to set a cookie on every request but only when strictly needed. Minimizing the duration of locks Tip: Acquire locks on shared resources just before you access them, and release them immediately after you are finished with them. By limiting the time each resource is locked, you minimize the time threads need to wait for resources to become available.

0
0
8025

Packt

08 Aug 2013

26 min read

Form customizations

Packt

08 Aug 2013

26 min read

(For more resources related to this topic, see here.) Forms are probably the most important visual element of the Dynamics CRM 2011 interface. To find the underlying data in every entity record, the user has to open the form. Dynamics CRM 2011 supports two types of forms: The main form : Dynamics CRM 2011 uses this form to allow the user to enter and view data within the Dynamics CRM 2011 web user interface as well as the Dynamics CRM 2011 within Microsoft Outlook interface. One main form per entity exists by default. However, multiple main forms can be created for an entity. Dynamics CRM 2011 supports role-based forms, which means separate forms can be visible depending on the security roles of the current user. Usually, multiple main forms are created when role-based forms have to be supported. The mobile form : Dynamics CRM 2011 uses this form when a user is accessing CRM from a mobile device that is compatible with HTML 4.0 using a URL such as <CRM_server> /m, where <CRM_server> is the path of Microsoft Dynamics CRM 2011 Server. A separate form for mobile devices is useful considering the limited space usually available on a mobile screen. A mobile form does not store data on a mobile device. If users try to access Dynamics CRM 2011 from an unsupported browser, they will be redirected to the mobile form. The following table outlines the browsers supported by Microsoft Dynamics CRM 2011: Browser Version / other requirements Internet Explorer IE7 (only for the on-premises version) IE 8, IE9 IE10 (desktop mode only) Mozilla Firefox Latest publicly released version running on Windows 8, Windows 7, Windows Vista, or Windows XP Google Chrome Latest publicly released version running on Windows 8, Windows 7, Windows Vista, or Windows XP Apple Safari Latest publicly released version running on Mac OS X 10.7 (Lion) or 10.8 (Mountain Lion) Detailed information about supported browsers can be found at http://technet.microsoft.com/en-us/library/hh699710.aspx. Dynamics CRM 2011 also supports special variants of the main form, as follows: The read-optimized form : Dynamics CRM 2011 has another type of form called the read-optimized form. Introduced in Update Rollup 7, this form is designed for the fast display of a record by disabling the ribbon and form scripts. This form displays the record in the read-only mode. Read-optimized forms are disabled by default and can be enabled by going to System | Administration | System Settings | Customization | Form Mode . Update Rollup 12 has introduced the following changes in read-optimized forms: The navigation pane for read-optimized forms is now enabled and the navigation pane can be expanded or collapsed. Support for web resources has been added. A new setting in the web resource properties, called Show this Web Resources in Read Optimized form , has been added. This setting must be enabled for the web resources to display in the read-optimized form. If the web resource depends on form resources, which are not available in a read-optimized form, we should not display it. Read-optimized forms honor all field-level security and role-based form definitions. If an entity has more than one form enabled, the read-optimized form uses the form that the user last used. The process-driven form : The December 2012 Service Update (Polaris update) of Dynamics CRM 2011 has introduced an enhanced read-optimized form, commonly known as the process-driven form for the Account, Contact, Lead, Opportunity, and Case entities. This new type of form is very useful, especially for touch devices, as the new form is designed to contain everything in one form; there is no need to open multiple pop ups. However, this new form type cannot be used for any entity other than the entities listed above. For the Account, Contact, Lead, Opportunity, and Case entities, in addition to the information form, there will be a new form with the same name as that of the entity. The <entity name> form will always display using the updated presentation, regardless of the settings for read-optimized forms. However, if read-optimized forms are enabled for the organization, the information form will also display using the updated presentation. These new forms are not available in an on-premises deployment of Microsoft Dynamics CRM 2011. Form editor We need to use a form editor to customize a form within Dynamics CRM 2011. The form layout definition is actually stored as an XML file called Form Xml in the SystemForm entity. The customization.xml file exported with an unmanaged solution contains the definition of the entity forms. Creating and customizing an entity main form Almost all the business entities have a customizable main form. The Activity entity does not have any form and some entity forms such as the Case Resolution entity form are not customizable. When a custom entity is created, one main and one mobile form are added automatically. In this recipe, we will focus our discussion on how to customize a main form. Getting ready Dynamics CRM 2011 introduced a flexible layout for form design. The following diagram outlines the typical main form layout within the Dynamics CRM 2011 system: The major visible components of a standard main form are as follows: Ribbon : This is the top area of the form. We cannot customize this using the form editor. Entity icon : This displays the Icon for Entity Form icon of the entity. It is a 32 x 32 pixel image and can be updated for an entity. Header and footer : The header and footer are two read-only areas of the form layout. These two sections remain static when a user scrolls through the form data displayed by the various tabs and sections. So any data that is required to be available to the user irrespective of any scrolling, can be included in these sections. Form selector : When an entity has multiple forms and the current user's security role has access to more than one form, the form selector is displayed. The user can use the form selector to choose a form from multiple forms available to them. Navigation : This section allows users to navigate to related records of the current record. We can add, modify, delete, or reorganize the link to the related entity records using the form editor. We can also include links to URLs or web resources by adding navigation links using the form editor. Form assistant : It helps when we set values for lookup fields. Dynamics CRM 2011 has introduced improved capabilities to filter data returned in the lookup dialog. Hence, the form assistant is no longer useful; the form assistant has been turned off for all except the following three entity forms: Case Product Service activity Tabs and sections : Tabs and sections allow grouping and laying out of controls in a form. A tab can contain multiple sections. Each form can have a maximum of 100 tabs. Tabs have a vertical collapse/expand feature. We will now take a look at the various form-body elements that can be added or associated with an entity form: Field : Each field represents an attribute of the entity. A field can be added to a form using the form editor and the form editor allows us to add the same field multiple times in a form. Each instance of a field in a form is known as a control . The appearance and behavior of a control is driven by the type and formatting options of the attribute as well as display and formatting properties set on the control, using the form editor. Tab and section : As previously discussed, tabs and sections are used for grouping the controls in the form. A tab can contain multiple sections within it. Each tab or section can be assigned a name. We can choose to display the name of the tab or section on the form or include a separator line at the top of the tab or section, underneath the name. A tab can have one column or two columns; when two columns are specified, the width of each column is a percentage of the width of the tab. A section, on the other hand, may have up to four columns and we can control the width available for control labels to be displayed in the section as well as how labels for controls in the section should be aligned. Spacer : The Spacer element provides extra space between fields and controls in the form. This is used to improve the control layout in a section. Sub-Grid : Sub-Grid allows us to display a list of records, charts, or both. The first four subgrids can be populated with data in a form when it loads. If more than four subgrids exist on a form, the remaining subgrids require some user or form script action to retrieve data. This is for performance optimization. IFRAME : This control provides the HTML iFrame element in the form. Using the control, we can host another web page within the Dynamics CRM 2011 entity form. The form editor provides the ability to set regular iFrame properties along with properties specific to Dynamics CRM 2011. Web Resource : This control displays a form-enabled web resource to be displayed on the page. A form-enabled web resource includes a web page (HTML), image (JPG, PNG, GIF, ICO), or Silverlight (XAP) resource. The web resource contents are hosted within Dynamics CRM 2011. Notes : If the entity uses notes and attachments, we can add the Notes control into the form. This control can only be added if the entity has Notes enabled in the entity definition. Navigation Link : This control is available only within the Navigation section of the form. This control allows us to add a link to an external URL or web resource. How to do it… In this recipe, we will first discuss how to create a new main form and then discuss the form-customization options. The customization steps can be carried out on any main form. The entity main form can be customized by carrying out the following tasks: Editing tabs Editing sections Editing fields Editing header and footer Adding subgrids Adding iFrames Adding web resources Editing the Navigation area Editing form properties Making the form non-customizable In this recipe, we will discuss all the previously stated tasks one after the other. Please follow these steps to customize the main form for an entity: Log in to the Dynamics CRM 2011 system as a system administrator or with a relevant security role. Navigate to Settings | Customizations | Solutions and change the view to Unmanaged Solutions , if not already selected. Then double-click on the unmanaged solution to open it. On the expanded Solution page, navigate to Components | Entities | <Entity> | Forms . The next step is to create a new main form; this can be done in two ways. We will discuss both of these here: Creating an entirely new main form : Go to New | Main Form in the actions toolbar. This will create a new form by copying the existing main form. When the new form pops up, click on the save button to save the form. Creating a new form from an existing form : Open the existing form by double-clicking on it. When the form launches, click on Save As in the top ribbon. When the Save As -- Webpage Dialog window pops up, provide data for the Name and Description fields of the new form. Finally, click on the OK button to save the new form as shown in the following screenshot: Any newly created main form will be assigned only to the system administrator and system customizer security roles by default. To customize a main form, open the form by double-clicking on it in the forms list. The next step is to discuss the editing of tabs in the form. Tabs are collapsible controls that can contain section controls. The following two points will demonstrate adding a new tab and editing tab properties: Adding a new tab in the form : Click on Body in the form ribbon and then click on the Insert tab in the form. In the Insert tab, under the Tab group, select One Column to create a one-column tab, or Two Columns to create a two-column tab: If we add a tab, Dynamics CRM 2011 will automatically add a section for each column. To remove any control in an entity form, use the Delete key on the keyboard. Alternatively, the Remove button in the ribbon can also be used. Editing tab properties : Select the tab control and then click on the Change Properties button in the form ribbon. The Tab Properties page will open with the following properties being modifiable: Tab property Description Under the Display tab Name The unique name of the tab. Label The display label for this tab. This text will appear on the form. Show the label of this tab on the Form This determines whether the label defined for this tab will be displayed on the form. Select this option to enable the display of the tab's label on the form. Expand this tab by default If selected, the tab control will be displayed in expanded mode by default. Visible by default If selected, the tab control will be visible by default in the form. Under the Formatting tab Select tab layout Choose between One Column and Two Columns to define the layout of the tab. Column 1 width If the Two Columns option is selected in the tab layout, we can specify the width of column 1 as a percentage. Column 2 width If the Two Columns option is selected in the tab layout, we can specify the width of column 2 as a percentage. The Events properties Scripts libraries can be linked to the tab. The scripts functions will be called on the TabStateChange event. Next we will see the editing of a section in a tab. A section contains fields in the form. The following two sections will demonstrate adding a section in a form and editing the section's properties: Adding a section in the form : Select the tab control where the new section is to be added and then click on the Insert tab in the form ribbon. Thereafter, click on One Column , Two Columns , Three Columns , or Four Columns under the Section group depending on whether a section with one, two, three, or four columns is to be added. Editing section properties : Select the section control and then click on the Change Properties button in the form ribbon. The Section Properties page will open and the following properties will be modifiable: Section property Description Under the Display tab Name The unique name of the tab. Label The display label for this tab. This text will appear on the form. Show the label of this section on the Form This determines whether the label defined for this section will be displayed on the form. Select this option to enable the display of the section's label on the form. Show a line at top of the section If selected, a divider line will be displayed underneath the name of the section. Width Specify the width of the label area of the fields in this field. The width must be set between 50 and 250 pixels. Visible by default If selected, the section control will be visible by default on the form. Lock the section of the Form If selected, the section would be locked in the form. Under the Formatting tab Layout Choose from among One Column, Two Columns, Three Columns, and Four Columns to define the layout of the section control. Field label alignment Select between the Left and Right alignments for the field labels in the section control. Next we will take a look at editing a field in the section: Adding a field in a section : Select the section where the field has to be added. Thereafter, find the field in the right-hand side Field Explorer pane. By default, the Field Explorer pane displays all unused fields in the form. If we want to add a field that is already used in the form, uncheck the Only show unused fields checkbox as shown in the following screenshot: After selecting the field in Field Explorer , move the field by pressing the left mouse button and drop the field in the intended column of the section. The red line on top of the column indicates that the column has been selected. Now drop the field on the selected column. Editing field properties : To edit the form-level properties of the field, select the field and then click on the Change Properties button in the form ribbon. Then the Field Properties pop up will open and the following properties can be modified: Field property Description Under the Display tab Label Here you can edit the display name of the field on the form. By default, the display name of the field will be displayed there, which can be edited to provide a new display name for the field on the form. Display Label on the form This determines whether the display name of the field is to be displayed in the form. Field is read-only This determines whether a field is to be read-only for the users in the form. Lock the field on the form This determines whether the field is to be locked on the form. Visible by default This determines the default visibility of the control in the form. Under the Formatting tab Layout This determines the width of this field on the form. The width of a field depends on the layout settings of the section it is in. The Details properties This tab displays the details of the field definition. Click on the Edit button to modify those properties of the field definition that can be modified. The Event properties Script libraries can be linked to the tab. The scripts' functions will be called on the OnChange event. If the field is of type Lookup (N:1 relationship with another entity), then there exists an additional set of properties in the Field Properties list. These properties can be set to save the user's time, find the appropriate parent record, or to restrict the user to select among a subset of records in the parent entity. The following form-level properties of the lookup field can be edited: Property name Description Turn off automatic resolutions in the field If this setting is disabled (not selected) and if a user enters a partial value for the lookup field and tabs away, Dynamics CRM 2011 will try to autopopulate the lookup field. Disable most recently used items for this field If this setting is disabled (not selected), Dynamics CRM 2011 will automatically provide a list of recently selected values for the user to choose from. This property is not supported for process-driven forms of Microsoft Dynamics CRM 2011 Online. Related Record Filtering This setting provides a way to limit the list of records that the user can choose from. The list under the Only show records where heading displays all the potential relationships that can be used to filter this lookup. Once a record is selected, the list under the Contains heading will display all relationships that connect the related entity (selected in the first list) to the target entity. Select the Allow users to turn off filter checkbox to provide users with the option to turn off the filter defined here. This makes it possible for them to view a wider range of records. Additional properties This setting controls how much search flexibility the user will have in terms of changing among various views and searching the record with a search box. Select the Display Search Box in lookup dialog checkbox if you want a search box to be available in the lookup. In the Default View list, select the default view for which results will be displayed in the lookup. Finally, choose the views we want users to have access to in the lookup, using the View Selector list. Adding a new entity field and then adding it to the form : A new field can also be created and then added to the entity from the form. To create a new field, click on the New Field button at the bottom of the Field Explorer pane. This will launch the new field pop up. Next we will delve into editing headers and footers. To edit the header or footer of the form, click on the Header or Footer button in the form ribbon and the section will be focused automatically. Then click on Change Properties in the ribbon. The Header Properties or Footer Properties page will pop up and we can edit the following settings: Header/footer property Description Under the Display tab Width Specify the width field label area here. The width must be set between 50 and 250 pixels. Lock the section of the Form This setting is selected by default and cannot be modified. This setting determines whether the section would be locked in the form or not. Under the Formatting tab Layout Here you can choose from among One Column, Two Columns, Three Columns, and Four Columns to define the layout of the header/footer control. Field Label Alignment Select from the Left (default), Right, or Center alignment for the field labels in the header/footer control. Field Label Position Select between Side (default) and Top to specify whether the field label in this section will be on the left-hand side or above the field. Fields can be added to the header or footer controls in the same way they are added in any section control in the form. Next we will look at how to add subgrids. The Sub-Grid control displays related entity records in the form body, using the following steps: Select the section control where the subgrid is to be added in the form. Then click on the Sub-Grid button under the Insert tab in the form ribbon. This will bring up the List or Chart Properties page, where we can specify the following properties of a subgrid: Subgrid property Description Under the Display tab Name The unique name of the subgrid control. Label The display text of the subgrid. This text will be displayed on the form. Display label on the Form Select to confirm that the Label text will be displayed on the form. Data Source This specifies the primary data source of the subgrid. The Records list allows us to select between Only Related Records (to set only entities having a relationship to the current entity) and All Record Types (to set all available entities). We can choose the related entity from the Entity list. This list content will vary based on the earlier list's selection. The Default View list allows us to choose which view is to be displayed in the subgrid. Display Search Box Select this setting to display the search box in the subgrid. Display Index Select this setting to display the alphabetic index record selector in the subgrid. This property is not supported for process-driven forms of Microsoft Dynamics CRM 2011 Online. View Selector Select this setting to display the view selector in the subgrid. This property is not supported for process-driven forms of Microsoft Dynamics CRM 2011 Online. Chart Options Select whether to display a chart selector along with a default chart or show only a specified chart in place of the subgrid. This property is not supported for process-driven forms of Microsoft Dynamics CRM 2011 Online. Under the Formatting tab Layout Choose from among One Column, Two Columns, Three Columns, and Four Columns to define the layout of the subgrid control. Number of Rows Select the maximum number of rows to be displayed in the subgrid control. The number of rows has to be between 2 and 250. Automatically expand to use available space Select this setting to enable automatic expansion of the subgrid to use available space in the form. iFrames or Inline Frames are HTML documents embedded inside the Dynamics CRM entity form. The following steps will guide you through adding an iFrame in the form: Select the section control where the iFrame is to be added in the form. Then click on the IFRAME button under the Insert tab in the form ribbon. This will bring up the Add an IFRAME page, where we can specify the following properties of an iFrame: iFrame property Description Under the General tab Name The unique name of the iFrame control. URL The URL of the HTML document to be displayed in the iFrame control. Pass record object-type code and unique identifier as parameters Select this option to pass contextual information entity object-type code and the record's unique identifier to the iFrame. Read more about this in the How it works... section of this recipe. Label Here, specify the display text for the iFrame. Display label on the Form Select this setting to display the label on the form. Restrict cross-frame scripting, where supported This checkbox is selected by default. We can remove this restriction only if we are certain that the HTML document/site we are using as the target of the iFrame can be trusted. Visible by default Select this setting to make the iFrame visible by default on the form. Under the Formatting tab Layout Choose from among One Column, Two Columns, Three Columns, and Four Columns to define the layout of the iFrame control. Number of Rows Select the maximum number of rows the iFrame control occupies on the form. The number of rows has to be between 1 and 40. Automatically expand to use available space Select this setting to enable automatic expansion of the iFrame control to use the available space in the form. Scrolling Select the scrolling option for the iFrame content display. Display Border Specify whether a border for the iFrame control is to be displayed. Web resources represent files that can be used to extend the Microsoft Dynamics CRM 2011 web application, such as HTML files, Image files, JScript library, and Silverlight applications. The following steps can be used to add a web resource in the form: Select the section control where the web resource is to be added in the form. Then click on the Web Resource button under the Insert tab in the form ribbon. This will bring up the Add Web Resource page, where we can specify the following properties of a web resource: Web resource property Description Under the General tab Web Resource Lookup to find a form-enabled web resource. Name The unique name for the web resource. Label Specify the display text for the web resource here. Display label on the Form Select this setting to display the label on the form. Visibility by default Select this setting to make the web resource visible by default on the form. Show this web resource in Read-Optimized Form Select this setting if the web resource is to be displayed in the read-optimized form. Under the Formatting tab Layout Choose from among One Column, Two Columns, Three Columns, and Four Columns to define the layout of the web resource control. Number of Rows Select the maximum number of rows the web resource control occupies on the form. The number of rows has to be between 1 and 40. Automatically expand to use available space Select this setting to enable automatic expansion of the web resource control to use the available space in the form. Scrolling Select the scrolling option for the web resource content display. Display Border Specify here whether a border for the web resource control is to be displayed. The Dependencies properties Select the fields from the Available fields list that are required by the web resource, and then click on the (add selected records) button to move the selected fields to the Dependent fields list. The navigation area displays entities that are related to the current entity. Each relationship has a Label property and in this navigation section this Label property is displayed by default. However, the display name for the related entity can be changed. This display name does not update the Label property of the relationship. In order to edit the navigation area, perform the following steps: Select the Navigation button in the form ribbon. The navigation section will be enabled. Then click on any relationship label and select Change Properties to edit the display text. This will bring up the Relationship Properties page. Modify the Label field here. Next we will edit the form properties; in order to do this, click on the Form Properties button in the form ribbon and the Form Properties page will pop up. The following properties can be edited there: Form property Description The Event properties Add or remove the JScript libraries that will be available for the form or field events. Under the Display tab Form Name The display name for the form. Modify this to rename the form. Description Specify a description for this form here. Show navigation items Select this setting to display the page navigation in the form. The Parameters properties Add query string parameters to be passed to the form. Click on the green plus sign to add a query string. We have to provide a Name value and select a Type value of the query string parameter. The Non- Event Dependencies properties Select the fields from the Available fields list that are required by any external, non-event scripts, and then click on the (add selected records) button to move the selected fields to the Dependent fields list. These fields will not be removable from the form. Lastly, making a form non-customizable restricts any future customization of the form. Therefore, to make a form non-customizable, perform the following steps: Select the Managed Properties button in the form ribbon. The Managed Properties of System Form: Form web page dialog will pop up. In this page, mark Customizable as False . After making any changes to an entity form, the form has to be saved and published. Use the Publish button in the form ribbon to publish the changes. How it works… Web resources and iFrames are not displayed using the Microsoft Dynamics CRM 2011 for Outlook reading pane, but iFrames are displayed in read-optimized forms. When the Pass record object-type code and unique identifier as parameters setting is enabled, iFrames allow the form to pass the following contextual parameters to itself: Parameter name Description typename The name of the entity. type This takes in the entity type code, which is an integer value to uniquely identify an entity in a specific organization. Id A GUID that represents a record. orgname The organization's name. userlcid The user's language code. orglcid The organization's language code. The list of entity type codes can be found at http://msdn.microsoft.com/en-us/library/gg328086.aspx. The key points about entity type codes are as follows: Type codes below 10,000 are reserved for out-of-the-box entities. Custom entities will have a type code greater than or equal to 10,000. Custom entities' type codes might change during solution import. Hence the type codes of a custom entity might be different in the development and test environments. The entity codes are stored in the Dynamics CRM database and can be retrieved from the EntityView table of the <OrganizationName>_MSCRM database.

0
0
7989

Packt

30 Dec 2014

13 min read

Middleware

Packt

30 Dec 2014

13 min read

0
0
7953

Packt

29 Jan 2010

9 min read

Basic Doctest in Python

Packt

29 Jan 2010

9 min read

Doctest will be the mainstay of your testing toolkit. You'll be using it for tests, of course, but also for things that you may not think of as tests right now. For example, program specifications and API documentation both benefit from being written as doctests and checked alongside your other tests. Like program source code, doctest tests are written in plain text. Doctest extracts the tests and ignores the rest of the text, which means that the tests can be embedded in human-readable explanations or discussions. This is the feature that makes doctest so suitable for non-classical uses such as program specifications. Time for action – creating and running your first doctest We'll create a simple doctest, to demonstrate the fundamentals of using doctest. Open a new text file in your editor, and name it test.txt. Insert the following text into the file: This is a simple doctest that checks some of Python's arithmeticoperations.>>> 2 + 24>>> 3 * 310 We can now run the doctest. The details of how we do that depend on which version of Python we're using. At the command prompt, change to the directory where you saved test.txt. If you are using Python 2.6 or higher, type: $ python -m doctest test.txt If you are using python 2.5 or lower, the above command may seem to work, but it won't produce the expected result. This is because Python 2.6 is the first version in which doctest looks for test file names on the command line when you invoke it this way. If you're using an older version of Python, you can run your doctest by typing: $ python -c "__import__('doctest').testfile('test.txt')" When the test is run, you should see output as shown in the following screen: What just happened? You wrote a doctest file that describes a couple of arithmetic operations, and executed it to check whether Python behaved as the tests said it should. You ran the tests by telling Python to execute doctest on the files that contained the tests. In this case, Python's behavior differed from the tests because according to the tests, three times three equals ten! However, Python disagrees on that. As doctest expected one thing and Python did something different, doctest presented you with a nice little error report showing where to find the failed test, and how the actual result differed from the expected result. At the bottom of the report, is a summary showing how many tests failed in each file tested, which is helpful when you have more than one file containing tests. Remember, doctest files are for computer and human consumption. Try to write the test code in a way that human readers can easily understand, and add in plenty of plain language commentary. The syntax of doctests You might have guessed from looking at the previous example: doctest recognizes tests by looking for sections of text that look like they've been copied and pasted from a Python interactive session. Anything that can be expressed in Python is valid within a doctest. Lines that start with a >>> prompt are sent to a Python interpreter. Lines that start with a ... prompt are sent as continuations of the code from the previous line, allowing you to embed complex block statements into your doctests. Finally, any lines that don't start with >>> or ..., up to the next blank line or >>> prompt, represent the output expected from the statement. The output appears as it would in an interactive Python session, including both the return value and the one printed to the console. If you don't have any output lines, doctest assumes it to mean that the statement is expected to have no visible result on the console. Doctest ignores anything in the file that isn't part of a test, which means that you can place explanatory text, HTML, line-art diagrams, or whatever else strikes your fancy in between your tests. We took advantage of that in the previous doctest, to add an explanatory sentence before the test itself. Time for action – writing a more complex test We'll write another test (you can add it to test.txt if you like) which shows off most of the details of doctest syntax. Insert the following text into your doctest file (test.txt), separated from the existing tests by at least one blank line: Now we're going to take some more of doctest's syntax for a spin.>>> import sys>>> def test_write():... sys.stdout.write("Hellon")... return True>>> test_write()HelloTrue Think about it for a moment: What does this do? Do you expect the test to pass, or to fail? Run doctest on the test file, just as we discussed before. Because we added the new tests to the same file containing the tests from before, we still see the notification that three times three does not equal ten. Now, though, we also see that five tests were run, which means our new tests ran and succeeded. What just happened? As far as doctest is concerned, we added three tests to the file. The first one says that when we import sys, nothing visible should happen. The second test says that when we define the test_write function, nothing visible should happen. The third test says that when we call the test_write function, Hello and True should appear on the console, in that order, on separate lines. Since all three of these tests pass, doctest doesn't bother to say much about them. All it did was increase the number of tests reported at the bottom from two to five. Expecting exceptions That's all well and good for testing that things work as expected, but it is just as important to make sure that things fail when they're supposed to fail. Put another way; sometimes your code is supposed to raise an exception, and you need to be able to write tests that check that behavior as well. Fortunately, doctest follows nearly the same principle in dealing with exceptions, that it does with everything else; it looks for text that looks like a Python interactive session. That means it looks for text that looks like a Python exception report and traceback, matching it against any exception that gets raised. Doctest does handle exceptions a little differently from other tools. It doesn't just match the text precisely and report a failure if it doesn't match. Exception tracebacks tend to contain many details that are not relevant to the test, but which can change unexpectedly. Doctest deals with this by ignoring the traceback entirely: it's only concerned with the first line—Traceback (most recent call last)—which tells it that you expect an exception, and the part after the traceback, which tells it which exception you expect. Doctest only reports a failure if one of these parts does not match. That's helpful for a second reason as well: manually figuring out what the traceback would look like, when you're writing your tests would require a significant amount of effort, and would gain you nothing. It's better to simply omit them. Time for action – expecting an exception This is yet another test that you can add to test.txt, this time testing some code that ought to raise an exception. Insert the following text into your doctest file (Please note that the last line of this text has been wrapped due to the constraints of the article's format, and should be a single line): Here we use doctest's exception syntax to check that Python iscorrectly enforcing its grammar.>>> def faulty():... yield 5... return 7Traceback (most recent call last):SyntaxError: 'return' with argument inside generator(<doctest test.txt[5]>, line 3) The test is supposed to raise an exception, so it will fail if it doesn't raise the exception, or if it raises the wrong exception. Make sure you have your mind wrapped around that: if the test code executes successfully, the test fails, because it expected an exception. Run the tests using doctest and the following screen will be displayed: What just happened? Since Python doesn't allow a function to contain both yield statements and return statements with values, having the test to define such a function caused an exception. In this case, the exception was a SyntaxError with the expected value. As a result, doctest considered it a match with the expected output, and thus the test passed. When dealing with exceptions, it is often desirable to be able to use a wildcard matching mechanism. Doctest provides this facility through its ellipsis directive, which we'll discuss later Expecting blank lines in the output Doctest uses the first blank line to identify the end of the expected output. So what do you do, when the expected output actually contains a blank line? Doctest handles this situation by matching a line that contains only the text <BLANKLINE> in the expected output, with a real blank line in the actual output. Using directives to control doctest Sometimes, the default behavior of doctest makes writing a particular test inconvenient. That's where doctest directives come to our rescue. Directives are specially formatted comments that you place after the source code of a test, which tell doctest to alter its default behavior in some way. A directive comment begins with # doctest:, after which comes a comma-separated list of options, that either enable or disable various behaviors. To enable a behavior, write a + (plus symbol) followed by the behavior name. To disable a behavior, white a – (minus symbol) followed by the behavior name. Ignoring part of the result It's fairly common that only part of the output of a test is actually relevant to determining whether the test passes. By using the +ELLIPSIS directive, you can make doctest treat the text ... (called an ellipsis) in the expected output as a wildcard, which will match any text in the output. When you use an ellipsis, doctest will scan ahead until it finds text matching whatever comes after the ellipsis in the expected output, and continue matching from there. This can lead to surprising results such as an ellipsis matching against a 0-length section of the actual output, or against multiple lines. For this reason, it needs to be used thoughtfully.

0
0
7899

article-image-extracting-data-using-dom-must-know

Packt

12 Sep 2013

5 min read

Extracting data using DOM (Must know)

Packt

12 Sep 2013

5 min read

(For more resources related to this topic, see here.) Getting ready This section will parse the content of the page at, http://jsoup.org. The index.html file in the project is provided if you want to have a fi le as input, instead of connecting to the URL. How to do it... The following screenshot shows the page that is going to be parsed: By viewing the source code for this HTML page, we know the site structure. The jsoup library is quite supportive of the DOM navigation method; it provides ways to find elements and extract their contents efficiently. Create the Document class structure by connecting to the URL. Document doc = Jsoup.connect("http://jsoup.org").get(); Navigate to the menu tag whose class is nav-sections. Elements navDivTag = doc.getElementsByClass("nav-sections"); Get the list of all menu tags that are owned by &#lt;a> . Elements list = navDivTag.get(0).getElementsByTag("a"); Extract content from each Element class in the previous menu list. for(Element menu: list) {System.out.print(String.format("[%s]", menu.html()));} The output should look like the following screenshot after running the code: The complete example source code for this section is placed at sourceSection02. The API reference for this section is available at: http://jsoup.org/apidocs/org/jsoup/nodes/Element.html How it works... Let's have a look at the navigation structure: html > body.n1-home > div.wrap > div.header > div.nav-sections > ul >li.n1-news > a The div class="nav-sections" tag is the parent of the navigation section, so by using getElementsByClass("nav-sections"), it will move to this tag. Since there is only one tag with this class value in this example, we only need to extract the first found element; we will get it at index 0 (first item of results). Elements navDivTag = doc.getElementsByClass("nav-sections"); The Elements object in jsoup represents a collection ( Collection<>) or a list (List<>); therefore, you can easily iterate through this object to get each element, which is known as an Element object. When at a parent tag, there are several ways to get to the children. Navigate from subtag <ul>, and deeper to each <li> tag, and then to the <a> tag. Or, you can directly make a query to find all the <a> tags. That's how we retrieved the list that we found, as shown in the following code: Elements list = navDivTag.get(0).getElementsByTag("a"); The final part is to print the extracted HTML content of each <a> tag. Beware of the list value; even if the navigation fails to find any element, it is always not null, and therefore, it is good practice to check the size of the list before doing any other task. Additionally, the Element.html() method is used to return the HTML content of a tag. There's more... jsoup is quite a powerful library for DOM navigation. Besides the following mentioned methods, the other navigation types to find and extract elements are also supported in the Element class. The following are the common methods for DOM navigation: Methods Descriptions getElementById(String id) Finds an element by ID, including its children. getElementsByTag(String c) Finds elements, including and recursively under the element that calls this method, with the specified tag name (in this case, c). getElementsByClass(String className) Finds elements that have this class, including or under the element that calls this method. Case insensitive. getElementsByAttribute(String key) Find elements that have a named attribute set. Case insensitive. This method has several relatives, such as: getElementsByAttribute Starting(String keyPrefix) getElementsByAttributeValue (String key, String value) getElementsByAttributeValue Not(String key, String value) getElementsMatchingText(Pattern pattern) Finds elements whose text matches the supplied regular expression. getAllElements() Finds all elements under the specified element (including self and children of children). There is a need to mention all methods that are used to extract content from an HTML element. The following table shows the common methods for extracting elements: Methods Descriptions id() This retrieves the ID value of an element. className() This retrieves the class name value of an element. attr(String key) This gets the value of a specific attribute. attributes() This is used to retrieve all the attributes. html() This is used to retrieve the inner HTML value of an element. data() This is used to retrieve the data content, usually applied for getting content from the <script> and <style> tags. text() This is used to retrieve the text content. This method will return the combined text of all inner children and removes all HTML tags, while the html() method returns everything between its open and closed tags. tag() This retrieves the tag of the element. Summary In this article we saw to extract data using DOM from an HTML page. It was seen that jsoup is quite a powerful library for DOM navigation. Resources for Article : Further resources on this subject: HTML5 Presentations - creating our initial presentation [Article] Building HTML5 Pages from Scratch [Article] JBoss Tools Palette [Article]

0
0
7865

article-image-working-simple-associations-using-cakephp

Packt

24 Oct 2009

5 min read

Working with Simple Associations using CakePHP

Packt

24 Oct 2009

5 min read

Database relationship is hard to maintain even for a mid-sized PHP/MySQL application, particularly, when multiple levels of relationships are involved because complicated SQL queries are needed. CakePHP offers a simple yet powerful feature called 'object relational mapping' or ORM to handle database relationships with ease.In CakePHP, relations between the database tables are defined through association—a way to represent the database table relationship inside CakePHP. Once the associations are defined in models according to the table relationships, we are ready to use its wonderful functionalities. Using CakePHP's ORM, we can save, retrieve, and delete related data into and from different database tables with simplicity, in a better way—no need to write complex SQL queries with multiple JOINs anymore! In this article by Ahsanul Bari and Anupom Syam, we will have a deep look at various types of associations and their uses. In particular, the purpose of this article is to learn: How to figure out association types from database table relations How to define different types of associations in CakePHP models How to utilize the association for fetching related model data How to relate associated data while saving There are basically 3 types of relationship that can take place between database tables: one-to-one one-to-many many-to-many The first two of them are simple as they don't require any additional table to relate the tables in relationship. In this article, we will first see how to define associations in models for one-to-one and one-to-many relations. Then we will look at how to retrieve and delete related data from, and save data into, database tables using model associations for these simple associations. Defining One-To-Many Relationship in Models To see how to define a one-to-many relationship in models, we will think of a situation where we need to store information about some authors and their books and the relation between authors and books is one-to-many. This means an author can have multiple books but a book belongs to only one author (which is rather absurd, as in real life scenario a book can also have multiple authors). We are now going to define associations in models for this one-to-many relation, so that our models recognize their relations and can deal with them accordingly. Time for Action: Defining One-To-Many Relation Create a new database and put a fresh copy of CakePHP inside the web root. Name the database whatever you like but rename the cake folder to relationship. Configure the database in the new Cake installation. Execute the following SQL statements in the database to create a table named authors, CREATE TABLE `authors` ( `id` int( 11 ) NOT NULL AUTO_INCREMENT PRIMARY KEY , `name` varchar( 127 ) NOT NULL , `email` varchar( 127 ) NOT NULL , `website` varchar( 127 ) NOT NULL ); Create a books table in our database by executing the following SQL commands: CREATE TABLE `books` ( `id` int( 11 ) NOT NULL AUTO_INCREMENT PRIMARY KEY , `isbn` varchar( 13 ) NOT NULL , `title` varchar( 64 ) NOT NULL , `description` text NOT NULL , `author_id` int( 11 ) NOT NULL ) Create the Author model using the following code (/app/models/authors.php): <?php class Author extends AppModel{ var $name = 'Author'; var $hasMany = 'Book';} ?> Use the following code to create the Book model (/app/models/books.php): <?phpclass Book extends AppModel{ var $name = 'Book'; var $belongsTo = 'Author';}?> Create a controller for the Author model with the following code: (/app/controllers/authors_controller.php): <?phpclass AuthorsController extends AppController { var $name = 'Authors'; var $scaffold;}?> Use the following code to create a controller for the Book model (/app/controllers/books_controller.php): <?php class BooksController extends AppController { var $name = 'Books'; var $scaffold; } ?> Now, go to the following URLs and add some test data: http://localhost/relationship/authors/ and http://localhost/relationship/books/ What Just Happened? We have created two tables: authors and books for storing author and book information. A foreign-key named author_id is added to the books table to establish the one-to-many relation between authors and books. Through this foreign-key, an author is related to multiple books, as well as, a book is related to one single author. By Cake convention, the name of a foreign-key should be underscored, singular name of target model, suffixed with _id. Once the database tables are created and relations are established between them, we can define associations in models. In both of the model classes, Author and Book, we defined associations to represent the one-to-many relationship between the corresponding two tables. CakePHP provides two types of association: hasMany and belongsTo to define one-to-many relations in models. These associations are very appropriately named: As an author 'has many' books, Author model should have hasMany association to represent its relation with the Book model. As a book 'belongs to' one author, Book model should have belongsTo association to denote its relation with the Author model. In the Author model, an association attribute $hasMany is defined with the value Book to inform the model that every author can be related to many books. We also added a $belongsTo attribute in the Book model and set its value to Author to let the Book model know that every book is related to only one author. After defining the associations, two controllers were created for both of these models with scaffolding to see how the associations are working.

0
0
7846

How-To Tutorials - Programming

Advanced Cypher tricks

Getting Started with Code::Blocks

Documenting Your Python Project-part1

Auto updating child records in Process Builder

Python Multimedia: Video Format Conversion, Manipulations and Effects

Salesforce CRM Functions

Python Libraries for Geospatial Development

The Command Line

Application Patterns

Tips and Tricks for Effectively Using ASP.NET

Trending Topics

Form customizations

Middleware

Basic Doctest in Python

Extracting data using DOM (Must know)

Working with Simple Associations using CakePHP

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access